M343 
Unit  8 


Mathematics:  A  Third  Level  Course 
The  Open  University 


Unit  8 

Birth  and  death  processes 


M343  APPLICATIONS  OF  PROBABILITY 


M343  APPLICATIONS  OF  PROBABILITY 

Mathematics:  A  Third  Level  Course 
The  Open  University 

9 


Unit  8 

Birth  and  death  processes 

Prepared  by  the  Course  Team 


CONTENTS 

Introduction  3 

1  Birth  and  death  processes  4 

1.1  Markov  processes  4 

1.2  Birth  and  death  processes  6 

1.3  The  Kolmogorov  forward  equations  7 

2  The  two-state  Markov  process  8 

2.1  The  model  9 

2.2  The  equilibrium  distribution  10 

3  The  simple  birth-death  process  13 

3.1  The  probability  generating  function  of  the  simple  birth-death 

process  1 3 

3.2  Some  properties  of  the  process  15 

4  More  about  the  simple  birth-death  process  18 

4.1  Embedded  processes  18 

4.2  Times  to  various  events  23 

5  The  effect  of  immigration  26 

5.1  The  immigration-death  model  (Audio)  26 

5.2  Immigration,  birth  and  death  32 

Objectives  35 

Appendix:  Solutions  to  questions  36 


The  Open  University  Press 


Statistics  tables 


The  recommended  book  of  statistics  tables  for  this  course  is  H.  R.  Neave, 
Elementary  Statistics  Tables  (George  Allen  &  Unwin,  1981).  In  this  unit,  these 
tables  are  referred  to  as  Neave. 


Unit  titles 


1 

Random  Processes 

9 

Queues 

2 

Events  in  Time 

10 

Epidemics 

3 

Patterns  in  Space 

11 

More  Population  Models 

4 

Branching  Processes 

12 

Genetics 

5 

Random  Walks 

13 

Renewal  Models 

6 

Markov  Chains 

14 

Diffusion  Processes 

7 

Birth  Processes 

15 

Time  Series 

8 

Birth  and  Death  Processes 

16 

Problems,  Problems,  Problems, 

The  Open  University  Press,  Walton  Hall,  Milton  Keynes. 

First  published  1988.  Reprinted  1992,  1996,  2000. 

Copyright  ©  1988  The  Open  University 

All  lights  reserved.  No  part  of  this  publication  may  be  reproduced,  stored  in  a  retrieval 
system  or  transmitted  in  any  form  or  by  any  means,  without  written  permission  from  the 
publisher. 

Designed  by  the  Graphic  Design  Group  of  the  Open  University. 

Printed  in  the  United  Kingdom  by  Page  Bros,  Norwich. 

ISBN  0  335  14292  3 

This  text  forms  part  of  the  correspondence  element  of  an  Open  University  Third  Level 
Course 

For  general  availability  of  supporting  material  referred  to  in  this  text,  please  write  to- 
Open  University  Educational  Enterprises  Limited,  12  Cofferidge  Close,  Stony  Stratford 
Milton  Keynes  MK1 1  1BY,  Great  Britain. 

Further  information  on  Open  University  courses  may  be  obtained  from:  The  Admissions 
Office,  The  Open  University,  P.O.  Box  48,  Milton  Keynes  MK7  6AB. 

1.4 


Introduction 


In  Unit  7  the  simple  birth  process  and  the  pure  death  process  were  studied.  In  the 
former  process  the  number  of  individuals  in  the  population  increases  with  time 
and,  in  the  cases  that  we  considered,  continues  to  increase  without  limit.  In  a  pure 
death  process  the  number  of  individuals  in  the  population  decreases  and  eventually 
the  population  becomes  extinct.  In  this  unit  we  shall  study  populations  in  which 
both  births  and  deaths  occur,  and  we  shall  also  study  populations  in  which,  in 
addition,  individuals  may  join  the  population  from  outside  at  any  time. 

There  are  more  possible  patterns  of  development  for  the  models  of  this  unit  than 
for  those  in  Unit  7.  When  births  and  deaths  are  both  possible,  the  size  of  the 
population  may  fluctuate  instead  of  only  increasing  or  only  decreasing.  It  is 
possible  that  the  population  size  will  decrease  to  zero  and,  if  no  new  individuals 
may  join  the  population,  it  will  then  become  extinct.  On  the  other  hand  the  size 
may  increase  indefinitely,  and  some  of  the  results  we  shall  obtain  are  reminiscent 
of  results  obtained  in  earlier  units  for  branching  processes  and  random  walks.  If,  in 
addition  to  births  and  deaths,  new  individuals  are  permitted  to  join  the  population 
at  any  time  then,  although  the  size  may  fall  to  zero,  zero  is  no  longer  an  absorbing 
state,  as  a  new  individual  will  eventually  arrive  and  restart  the  population.  We 
shall  investigate  the  behaviour  of  such  a  process:  whether  the  population  tends  to 
increase  unboundedly  with  time  or  whether  the  distribution  of  the  population  size 
approaches  an  equilibrium. 

In  Section  I  we  consider  the  general  theoretical  properties  of  models  in  which 
increases  and  decreases  may  both  occur.  Many  of  these  properties  also  apply  to 
processes  that  will  be  discussed  in  later  units.  In  Section  2  we  study  a  special  case 
in  which  the  process  has  only  two  possible  states;  this  can  be  thought  of  as  a 
model  for  a  population  whose  size  is  either  0  or  1.  This  process  provides  an 
appropriate  model  for  various  practical  situations:  for  instance,  for  a  machine 
which  is  either  working  or  broken  down  and  under  repair,  or  for  a  telephone 
which  is  either  in  use  or  idle.  In  the  case  of  the  machine,  breakdown  is  thought  of 
as  a  death  and  completion  of  repair  as  a  birth.  This  model  has  some  interesting 
properties,  and  the  methods  introduced  here  are  used  again  later  in  the  unit. 

The  simple  birth-death  process  is  developed  in  Section  3,  and  its  main  properties 
are  derived.  In  this  process  births  and  deaths  are  modelled  in  the  same  way  as  in 
Unit  7,  so  the  model  is  a  very  simple  one  and  is  not  applicable  to  human 
populations.  (Some  more  realistic  models  for  human  populations  are  described  in 
Unit  11.)  However,  it  can  be  applied  successfully  to  primitive  organisms  such  as 
bacteria,  yeast  or  mould  spores.  We  discuss  this  model  further  in  Section  4.  and 
some  of  the  results  from  our  previous  work  on  random  walks  and  branching 
processes  are  used  to  obtain  results  for  this  process.  In  Section  5  the  effect  of 
including  immigration  in  the  model  is  studied.  Subsection  5.1  contains  an  audio- 
tape  session  and  deals  with  a  model  for  immigration  and  death.  And  Subsection 
5.2  combines  all  the  results  of  the  unit  by  considering  a  process  with  birth,  death 
and  immigration.  This  subsection  contains  no  new  ideas  and  is  presented  as  a 
series  of  questions.  It  would  be  quite  possible  to  omit  this  subsection  on  first 
reading  and  study  it  later  when  you  are  revising  Units  7  and  8. 

As  in  Unit  7,  probability  generating  functions  will  be  used,  and  the  procedure  for 
solving  Lagrange’s  equation,  which  was  introduced  in  that  unit,  will  also  be  used. 

So  Unit  1  is  essential  preparation  for  this  unit.  To  give  you  further  practice  at 
applying  the  methods  of  Unit  7,  most  of  the  work  involving  Lagrange’s  equation  is 
included  as  questions;  this  work  is  in  Sections  3  and  5. 

When  you  study  Section  4,  you  will  need  to  use  some  of  the  results  for  the 
gambler’s  ruin  problem  of  Unit  5,  Section  2.  And  you  should  be  able  to  obtain  the 
probability  of  extinction  for  a  Galton- Watson  branching  process  as  in  Section  4 
of  Unit  4. 

Sections  1  and  2  are  fairly  short.  Section  3  is  also  quite  short,  but  it  contains  a  lot 
of  questions.  Section  5  is  the  longest  and  will  take  you  most  study  time  if  you 
decide  to  study  it  all  at  once.  However,  it  contains  no  new  concepts  but  quite  a 
few  questions. 

There  is  no  video  component  associated  with  this  unit. 


1  Birth  and  death  processes 

In  this  unit  we  continue  our  study  of  stochastic  processes  developing  in  continuous 
time.  In  particular,  we  discuss  processes  {X(t);  t  >  0}  in  which  the  random  variable 
X(t)  takes  only  discrete  values;  in  fact,  in  all  the  specific  models  discussed  in  this 
unit,  X(t)  takes  only  non-negative  integer  values.  The  processes  considered  are 
examples  of  a  class  of  stochastic  processes  known  as  Markov  processes.  Although 
the  theory  of  Markov  processes  in  continuous  time  was  not,  in  fact,  developed  by 
Markov,  his  name  has  been  given  to  these  continuous-time  processes  since  they  are 
the  continuous-time  analogue  of  Markov  chains  in  discrete  time. 

In  Subsection  1.1  a  Markov  process  in  continuous  time  is  defined  and  some 
general  properties  of  this  type  of  process  are  discussed.  And  in  Subsections  1.2  and 
1 .3  a  class  of  Markov  processes  known  as  birth  and  death  processes  is  discussed, 
and  some  notation  and  basic  results  are  developed  that  will  be  used  in  the 
remaining  sections  of  the  unit. 


1.1  Markov  processes 

Roughly  speaking,  a  Markov  process  is  a  process  in  which  the  probabilities  of 
future  behaviour  patterns  depend  only  on  the  present  state  of  the  process  and  are 
independent  of  its  past  history.  This  property  of  a  process  is  called  the  Markov 
property.  In  Unit  <5,  we  discussed  the  Markov  property  for  a  stochastic  process  in 
discrete  time,  and  a  Markov  chain  was  defined;  a  Markov  chain  is  a  Markov 
process  in  discrete  time.  Before  giving  a  corresponding  definition  for  a  process  in 
continuous  time,  we  shall  consider  briefly  three  processes  you  have  met  before,  two 
which  are  Markov  processes  and  one  which  is  not. 


Example  1.1  The  Poisson  process 


The  Poisson  process  is  the  simplest  example  of  a  Markov  process  in  continuous 
time.  In  a  Poisson  process,  {X(t);  t  >  0}  is  the  random  process  representing  the 
number  of  events  that  have  occurred  by  time  t.  If  the  number  of  events  observed 
to  have  occurred  by  some  time  tl  is  known,  then  the  probability  of  any  future 
pattern  of  events  can  be  obtained.  For  example,  if  X(tt)  =  x  then,  for  t  >  tXi  the 
probability  that  X(t)  =  x  is  equal  to  the  probability  that  no  further  events  have 
occurred  by  time  t,  and  this  is  where  a  is  the  rate  of  occurrence  of  events. 

And  the  probability  that  X(tx  +  t2)  =  x  +  y  is  equal  to  the  probability  that  a 
further  y  events  occur  in  the  time  interval  +  *2];  and  this  is  e~Xt2{Xt2Y/y\. 
Results  like  these  depend  on  Postulate  III  of  the  Poisson  process,  which  states  that 
events  before  time  t  are  independent  of  events  after  time  t.  It  follows  from  this 
postulate  that,  if  the  precise  times  at  which  the  x  events  occurred  were  known,  this 
would  not  make  any  difference  to  the  calculation  of  probabilities  about  events 
after  time  fj.  So  this  postulate  is  an  expression  of  the  Markov  property  of  the 
Poisson  process.  □ 


In  a  Poisson  process  with  rate  A,  the 
number  of  events  that  occur  in  any 
interval  of  length  u  has  a 
Poisson! /.u)  distribution. 


Example  1.2  The  simple  birth  process 

The  simple  birth  process  is  another  example  of  a  Markov  process.  For,  if  the 
population  is  known  to  be  of  size  x  at  a  particular  time  t,  so  that  X(t)  =  x,  then 
the  probability  of  a  birth  in  the  interval  [t,t  +  5 1]  is  pxSt  +  o(St),  and  this 
depends  only  on  x,  the  state  of  the  process  at  time  t,  not  on  the  behaviour  of  the 
process  before  time  t.  □ 

Example  1.3  Regularly  spaced  events 

Now  consider  a  process  in  which  events  occur  regularly  with  a  lime  interval  of 
length  a  between  events,  but  suppose  that  the  time  at  which  the  process  started  is 
unknown.  If  it  is  known  that  x  events  have  occurred  by  time  t,  this  information  is 
not  sufficient  to  specify  the  future  behaviour  of  the  process.  But  if,  in  addition,  it  is 
known  at  what  time  in  the  interval  [f  -  a,  f]  the  last  event  occurred,  then  the 
future  behaviour  of  the  process  is  specified  precisely.  So  the  probabilities  of  future 
behaviour  patterns  depend  on  the  past  history  of  the  process  as  well  as  the  present 
state;  and  therefore  this  process  is  not  a  Markov  process.  □ 
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This  is  Condition  (1.1)  of  Unit  6. 


Tn  Unit  6,  a  discrete-valued  process  { Xn ;  n  =  0,1,  ...}  in  discrete  time  is  said  to 
have  the  Markov  property  if,  for  any  states  i  and  j,  and  any  n>  0, 

p(xn+ 1  =j\X„  =  i  and  A)  =  P{Xn+l  =j \Xn  =  i), 
where  A  is  any  event  depending  only  on  {Xn.ltXn_2i  ...,  *0}.  That  is,  a  process 
has  the  Markov  property  if  the  probability  of  any  future  behaviour  of  the  process, 
when  its  current  state  is  known,  is  not  altered  by  any  additional  information  about 
its  past  behaviour. 

The  definition  of  a  Markov  process  in  continuous  time  is  based  on  a  similar 
condition.  Suppose  that,  at  any  time  t  >  0,  a  process  is  represented  by  the  random 
variable  X(t),  which  has  a  discrete  state  space.  Then  the  process  {X(t);  t  >  0}  is  a 
Markov  process  if,  for  any  sequence  of  times  tut2,  ...,  tn,u,t,  where 
0  <  f !  <  t2  <  ...  <  tn  <  u  <  t, 

P(X(t)  =  j | X(u)  =  UXitJ  =  iuX(t2)  =  i2, ...,  X(tn)  =  in) 

=  P(X(t)=j\X(u)  =  i),  (U) 

for  all  states  iui2,...,  in,i,j  in  the  state  space.  That  is,  for  a  Markov  process,  if  the 
state  of  the  process  is  known  at  some  time  u,  then  any  additional  information 
about  the  behaviour  of  the  process  before  time  u  does  not  affect  the  probability  of 
any  future  behaviour  pattern. 

Question  1.1  Which  of  the  following  processes  are  Markov  processes? 

(i)  A  pure  death  process 

(ii)  A  non-homogeneous  Poisson  process  in  which  the  probability  that  an  event 
occurs  in  the  interval  [t,t  +  <5t]  is  X(t)St  +  o(St) 

(iii)  A  P°int  process  in  which  the  time  interval  between  events  has  a  uniform 

distribution,  U(  1,3)  □ 

Transition  probabilities 

The  probability  P(X(t)  =j\X(u)  =  i )  is  sometimes  called  a  transition  probability.  In 
general,  it  depends  on  i,  j,  u  and  t,  so  to  demonstrate  this  dependence,  it  is  denoted 
by  Pij{u,t);  that  is, 

Pi.M,  t)  =  p(x(t)=j\x(u)  =  i). 

However,  for  many  Markov  processes,  the  time-dependence  of  the  transition 
probabilities  relates  only  to  t  -  u,  the  length  of  the  interval  [u,  t],  so  that,  for  any 
u ,  t  such  that  0  <  u  <  t, 

Pi,M,  t)  =  Pij( 0,  t  -  u), 

for  all  states  i  and  j.  In  this  case  the  Markov  process  is  said  to  be  homogeneous. 

The  Poisson  process  and  the  simple  birth  process  are  both  homogeneous  Markov 
processes. 

Suppose  that  a  homogeneous  Markov  process  is  in  state  i  at  time  t  =  0  and  that  it 
is  still  in  this  state  after  u  minutes.  What  is  the  probability  that  it  will  remain  in 
state  i  for  at  least  a  further  t  minutes?  If  T  is  the  time  until  the  first  change  of  state 
occurs,  then  the  probability  we  require  is  P(T>  u  +  t\  T>  u).  Now  for  a 
homogeneous  Markov  process,  the  (conditional)  probability  that  the  process 
remains  in  state  i  for  the  interval  [ u,u  +  t]  is  equal  to  the  (unconditional) 
probability  that  it  remains  in  state  i  for  an  interval  of  length  t.  So 
P(T>u  +  t\T>u)  =  P{T>t). 

In  other  words,  the  random  variable  T  has  the  memoryless  property  and  hence  it 
has  an  exponential  distribution. 

To  find  the  required  probability  we  would  need  additional  information  about  the 
process  so  that  we  could  find  the  parameter  of  this  exponential  distribution,  and 
this  we  do  not  have.  However,  in  attempting  to  find  this  probability  we  have 
obtained  a  very  useful  general  result  for  homogeneous  Markov  processes.  The  fact 
that  the  times  between  consecutive  events  in  a  homogeneous  Markov  process  are 
exponentially  distributed  is  a  fundamental  property  of  a  Markov  process,  and, 
moreover,  it  can  be  used  as  the  basis  for  a  definition.  Since  most  of  the  processes 
that  we  study  in  this  course  are  homogeneous  Markov  processes,  this  property  will 
be  used  on  many  occasions;  so  to  emphasize  its  importance,  it  is  stated  overleaf. 


Condition  (1.1)  is  the  Markov 
property  for  a  continuous-time 
process. 


A  homogeneous  process  is  the 
continuous-time  analogue  of  a 
discrete-time  process  with  stationary 
transition  probabilities. 
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A  fundamental  property  of  a  homogeneous  Markov  process  in  continuous  time 

The  time  intervals  between  consecutive  events  in  a  homogeneous  Markov 
process  in  continuous  time  are  exponentially  distributed. 


The  Chapman-Kolmogorov  equations 

Finally,  in  this  subsection,  we  shall  derive  a  set  of  equations  which  is  the 
continuous-time  analogue  of  the  Chapman-Kolmogorov  equations  for  a  Markov 
chain. 

Suppose  that  a  Markov  process  is  in  state  i  at  time  u,  and  that  we  want  to 
calculate  the  transition  probability  piyj(u,  t).  At  any  intermediate  time  v,  the  process 
must  be  in  some  state  k:  the  process  changes  from  being  in  state  i  at  time  u  to 
being  in  state  k  at  time  v  and  then  state  j  at  time  t.  Because  of  the  Markov 
property,  the  probability  of  the  change  from  k  to  j  does  not  depend  on  i,  and 
hence,  for  u  <  v  <  t, 

Pijv,  0  =  z  PiA  w,  V)  pkJ(v,  t),  (1.2) 

k 

where  the  summation  is  over  all  states  k  in  the  state  space.  These  equations  are 
known  as  the  Chapman-Kolmogorov  equations;  the  relation  (1.2)  holds  for  all  times 
u,v,t  such  that  u  <  v  <  t,  and  for  all  states  i  and  j.  The  Chapman-Kolmogorov 
equations  are  used  to  obtain  the  differential-difference  equations  for  a  Markov 
process. 


1.2  Birth  and  death  processes 

The  Poisson  process,  the  simple  birth  process  and  the  pure  death  process,  which 
were  studied  in  Unit  7,  are  all  examples  of  Markov  processes.  In  the  Poisson 
process  and  the  simple  birth  process  the  only  change  possible  in  a  small  interval  of 
time  is  an  increase  of  one— technically  known  as  a  birth.  And  in  the  pure  death 
process  the  only  change  possible  is  a  decrease  of  one — a  death.  In  this  unit  we 
study  Markov  processes  in  which  both  births  and  deaths  may  occur;  these  are 
known  as  birth  and  death  processes. 

In  a  birth  and  death  process,  the  random  variable  X(t)  represents  the  size  of  some 
population  at  time  t.  If  A(t)  =  x  then  the  possible  changes  in  the  interval  [t,t  -I-  St] 
are  a  birth,  increasing  the  population  size  to  x  +  1  at  time  t  +  St,  and  a  death, 
reducing  the  size  to  x  -  1  at  time  t  +  St,  and  these  have  probabilities 
Px,x+  i(M  +  <>t)  and  pXtX~x{t,t  +  <5t).  Since  the  process  is  a  Markov  process,  these 
probabilities  depend  only  on  x,  t  and  St;  and  so  we  may  write 

Px,x+i(t,t  +  St)  =  fix  St  +  o(St),  (1.3a) 

Px,x-i(t,t  +  St)  =  vx  St  +  o(St),  (1.3b) 

where  fix  and  v*  are  functions  of  x  and  t;  fix  is  called  the  overall  birth  rate  of  the 
process  and  vx  is  the  overall  death  rate.  Since  the  population  size  must  be  a  non¬ 
negative  integer,  the  death  rate  v0  is  defined  to  be  zero;  then  Equations  (1.3a)  and 

(1.3b)  hold  for  x  =  0, 1,  ...  .  The  probability  that  two  or  more  events  occur  in  the 

interval  [r,  t  +  St ]  is  o(<5t),  and  hence  the  probability  that  no  change  occurs  in 
[t,  t  +  St]  is  given  by 

Px,x(t,  t  +  St)  =  1  -  fix  St  -  vx  St  +  o(St).  (1.4) 

Note  that  if  both  a  birth  and  a  death  occur  in  [t,  t  +  Si],  the  population  size 
returns  to  x  at  the  end  of  the  interval;  the  probability  that  two  events  occur  is 
o(St)  and  this  is  included  in  the  probability  above. 

For  the  simple  birth  process  discussed  in  Unit  7,  each  individual,  independently  of 
all  other  individuals,  has  probability  fiSt  +  o(St )  of  giving  birth  to  one  new 
individual  in  any  interval  of  length  St.  Hence,  if  there  are  x  individuals  in  the 
population  at  time  t,  the  probability  of  a  birth  occurring  in  the  time  interval 
[t,t  4-  <5f]  is  fix  St  +  o(<5t).  So  fix,  the  overall  birth  rate,  is  equal  to  fix.  Since  no 
deaths  occur  in  this  process,  v*  =  0. 


Unit  6,  Equation  (1.8) 


fix  and  vx  may  be  functions  of  x 
only,  or  of  t  only,  or  of  both  x 
and  t. 
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Question  1.2  Find  the  values  of  (Jx  and  v*  for  the  pure  death  process.  □ 

Question  1.3  What  processes  are  represented  by  the  following  overall  birth  and 
death  rates? 

(i)  =  A,  vx  =  0. 

(ii)  /3X  =  A(t),  vx  =  0.  □ 

The  main  points  that  have  been  discussed  concerning  birth  and  death  processes  in 
general  are  summarized  below. 


A  birth  and  death  process  is  a  Markov  process  {X(t);  t  >  0}  in  which  X(t) 
takes  non-negative  integer  values  only,  and  the  only  changes  possible  in  any 
small  interval  [t,t  +  Sf]  are  an  increase  of  one  (a  birth)  and  a  decrease  of  one 
(a  death).  For  x  =  0, 1,  ...,  these  changes  have  probabilities 
Px,x+i(t,t  +  St)  =  pxdt  +  o(St), 

Px,x-i(t,t  +  St)  =  vxSt  +  o(8t),  (L3) 

where  (3X,  the  overall  birth  rate,  and  v„  the  overall  death  rate,  are  functions  of 
x  and  t;  v0  is  defined  to  be  zero.  The  probability  that  two  or  more  events 
occur  in  any  small  interval  is  o(3t),  and  hence  for  x  =  0, 1, .. ., 

PxA1’*  +  St)  =  1  -  (]xSt  —  vx  St  +  o(St).  (14) 


All  the  birth  and  death  processes  that  we  study  from  now  on  in  this  course  are 
homogeneous,  so  that  px  and  v*  are  independent  of  time.  Moreover,  for  the  last 
three  sections  of  this  unit,  and  in  the  next  one,  the  overall  birth  and  death  rates 
take  the  form  of  linear  functions  of  x;  that  is, 


Px  =  P*  +  A,  v*  =  vx  +  £.  (1  _5) 

The  overall  birth  rate  consists  of  two  terms:  /?x,  where  /?  is  the  individual  birth 

rate— that  is,  the  rate  at  which  each  individual  gives  birth  to  new  individuals _ 

and  A,  the  immigration  rate.  The  overall  death  rate,  vx,  also  consists  of  two  terms: 
vx,  where  v  is  the  individual  death  rate— so  that  the  lifetime  of  each  individual  has 
an  exponential  distribution  with  mean  1/v — and  e,  which  is  known  as  the 
emigration  rate.  In  any  small  interval  of  length  St  there  is  a  probability  of 

eSt  +  °(St)  that  one  ^dividual  leaves  the  population,  and  this  is  independent  of 
population  size. 


The  models  developed  in  Unit  7  include  several  in  which  vx,  the  overall  death  rate 
is  zero,  and  also  the  pure  death  process  for  which  0X  =  0,  vx  =  vx.  In  Section  2  of 
this  unit  we  study  the  simplest  model  in  which  both  births  and  deaths  can  occur: 
fl0  =  A,  Vl  =  v  and  all  other  (Jx,  vx  are  zero.  In  Sections  3  and  4  a  birth  and  death 
process  is  studied  in  which  fix  =  /?x,  v*  =  vx;  this  process  is  known  as  the 
simple  birth-death  process.  And  in  Section  5  this  model  is  extended  to  include 
immigration,  so  that  (jx  =  fix  +  A.  Emigration  is  not  considered  in  this  unit; 
however,  it  is  important  in  Unit  9,  where  various  models  for  queues  are  described. 


1.3  The  Kolmogorov  forward  equations 

For  a  birth  and  death  process,  {X(t);  t  >  0},  X(t)  is  the  size  of  the  population  at 
time  t.  Usually,  the  size  of  the  population  is  given  at  time  t  =  0,  so  to  find  the 
distribution  of  X(t\  we  must  calculate  the  probabilities  pix{0,t).  For  simplicity,  the 
zero  will  be  omitted  from  piiX(0,  t),  and  we  shall  write 

PiJt)  =  P(X(t)  =  x\X(0)  =  i) 

for  the  probability  that  the  population  size  is  x  at  time  t,  given  that  there  were  i 
individuals  at  the  start. 

We  shall  proceed  as  we  did  in  Unit  7  for  the  Poisson  process  and  the  simple  birth 
process,  the  first  stage  being  to  obtain  the  differential-difference  equations  satisfied 
by  the  probabilities  pitX(t). 
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Applying  the  Chapman-Kolmogorov  equations  (1.2),  with  0,  t  and  t  +  St  replacing 
u,  v  and  t  respectively,  we  have 

Pijt  +  <5t)  =  £  PkJU  t  +  St).  (1.6) 

k 

Now  the  only  values  of  k  for  which  the  transition  probability  pk_x(t,  t  +  St)  is  non- 
negligible  are  k  =  x  -  1,  k  =  x  +  1  and  k  =  x;  and,  from  Equations  (1.3)  and  (1.4), 
we  have 


px-i,x(t,t  +  St)  =  px-xSt  +  o(<5r), 

Px+  i,*(M  +  St)  =  vx+1St  +  o(St), 

Px,x(f»  t  +  St)  =  1  -  pxSt  -  vxSt  +  o(St). 

These  are  the  transition  probabilities  corresponding  to  the  situations  where  the 
population  size  is  x  —  1  at  time  t  and  a  birth  occurs  in  the  interval  [r,  t  +  St],  the 
size  is  x  +  1  at  time  t  and  a  death  occurs,  and  the  size  is  x  at  time  t  and  no 
change  occurs. 


Substituting  these  probabilities  in  Equation  (1.6)  leads  to 

Pijt  +  St)  =  pUx-x(t)px-ySt  +  Pi,x+1(t)vx+1St  +  Pijt)(  1  -  PxSt  -  vxSt)  +  o(St). 
Rearranging  this,  we  obtain,  for  x  =  1,2,  ..., 


Pijt  +  St)  -  pi>x(t) 
St 


Px-iPi.x-i(t)  +  vx+1pLx+1{t)  -  (px  +  vx)PiJt)  + 


o(St) 
St  ' 


The  death  rate  v0  has  already  been  defined  to  be  zero,  so  provided  x (r)  is 
defined  to  be  zero,  this  equation  also  holds  for  x  =  0. 


Usually  i,  the  size  of  the  population  at  t  =  0,  is  given  and,  for  simplicity,  i  is 

omitted  from  pitX(t).  So,  as  in  Unit  7,  px{t)  is  used  to  represent  the  probability  that  p_i(t)  is  defined  to  be  zero, 
the  population  contains  x  individuals  at  time  t  assuming  ,Y(0)  =  i,  some  known 
value.  Writing  px(t)  for  pix(t)  and  letting  St-*  0  in  the  equation  above  leads  to  the 
following  differential-difference  equations: 

jtPM  =  Px-iPx-i(t)  +  Vx+iPx+i(t)-(Px  +  vx)px(t),  x  =  0,l, ....  (1.7) 

These  are  the  differential-difference  equations  for  the  general  birth  and  death 
process;  they  are  known  as  the  Kolmogorov  forward  equations.  By  identifying  px 
and  vx,  the  overall  birth  and  death  rates,  they  can  be  used  to  write  down  directly 
the  differential-difference  equations  for  any  specific  process  of  this  type. 


Question  1.4  Use  the  Kolmogorov  forward  equations  (1.7)  to  write  down  the 
differential-difference  equations  for 

(i)  the  Poisson  process; 

(ii)  the  simple  birth  process.  □ 


2  The  two-state  Markov  process 


One  of  the  simplest  Markov  processes  is  one  in  which  there  are  only  two  possible 
states.  This  might  arise  as  a  model  in  various  situations.  For  example,  suppose 
that  a  machine  runs  for  an  exponentially  distributed  time  before  breaking  down, 
the  repair  then  takes  a  lime  which  is  also  exponentially  distributed — in  general 
with  a  different  mean — and  the  machine  then  starts  to  run  again.  The  two  states 
are  ‘working’  and  ‘under  repair’.  Another  practical  situation  which  may  be 
modelled  in  this  way  is  the  use  of  a  telephone  line  that  is  either  free  or  busy.  Elere 
it  must  be  assumed  that  callers  do  not  ‘queue’;  that  is,  if  a  caller  finds  the  line  busy 
he  does  not  try  again.  The  behaviour  of  an  electron  flowing  in  a  semi-conductor 
has  also  been  modelled  as  a  two-state  Markov  process.  The  electron  is  free  to  flow 
for  a  random  length  of  time,  which  is  exponentially  distributed;  it  is  then  trapped 


for  an  exponentially  distributed  time,  after  which  it  is  free  to  move  again.  The 
model  has  even  been  applied  to  the  behaviour  of  cows  who  spend  their  days 
alternately  grazing  and  chewing  the  cud. 

In  Subsection  2.1  the  model  for  a  two-state  Markov  process  is  described  and  it  is 
shown  that  the  two-state  Markov  process  may  be  regarded  as  a  birth  and  death 
process.  The  Kolmogorov  forward  equations  are  used  to  obtain  forward  equations 
for  this  process,  and  these  equations  are  solved.  In  Subsection  2.2,  we  investigate 
the  behaviour  of  the  process  after  it  has  been  running  for  a  long  time. 


2.1  The  model 

The  process  has  two  states,  labelled  0  and  1.  For  the  machine  example,  the  process 
is  in  state  1  when  the  machine  is  working,  and  in  state  0  when  it  is  being  repaired. 
The  time  spent  in  each  state  has  an  exponential  distribution:  the  length  of  time 
spent  at  each  visit  to  state  0  is  distributed  as  M(X),  and  the  length  of  time  spent  at 
each  visit  to  state  1  is  distributed  as  M(v). 

Figure  2.1  shows  a  possible  realization  of  the  process  in  which  the  mean  time 
spent  at  each  visit  to  state  1  is  2  units  and  the  mean  time  for  state  0  is  1  unit. 


Figure  2.1  Simulation  of  a  two-state  Markov  process  with  X  =  1,  v  =  \ 


This  process  might  represent,  for  example,  the  behaviour  of  a  machine  which  runs 
for  a  mean  time  of  two  hours  before  breaking  down,  and  has  a  mean  repair  time 
of  one  hour. 

Question  2.1  Suppose  that  a  machine  works  for  a  time  that  is  exponentially 
distributed,  M(v),  and  then  breaks  down,  its  repair  time  being  exponentially 
distributed,  M(k).  Suppose  further  that  at  the  instant  that  it  is  first  observed,  the 
machine  is  working. 

(i)  Write  down  the  mean  and  variance  of  Tx ,  the  total  time  to  elapse  until  the 
machine  has  broken  down  once  and  its  repair  has  been  completed. 

(li)  Write  down  the  mean  and  variance  of  Wn,  the  time  to  elapse  until  the  machine 
has  broken  down  n  times  and  been  repaired  n  times.  □ 

The  two-state  Markov  process  as  a  birth  and  death  process 

The  two-state  Markov  process  is  a  simple  example  of  a  birth  and  death  process. 

To  see  that  this  is  so,  consider  a  birth  and  death  process  with  overall  birth  and 
death  rates,  fix  and  vx,  given  by 

fio  =  K  fix  =  0  for  X  =  1,2, 

iq  =  v,  v*  =  0  for  x  =  2, 3,  ....  J  (2-1) 

The  only  possible  events  are  a  ‘birth’  to  increase  the  size  from  0  to  1,  and  a  ‘death’ 
to  decrease  the  size  from  1  to  0.  These  events  correspond,  respectively,  to  a  change 
from  state  0  to  state  1  when  the  repair  of  the  machine  is  completed,  and  a  change 

from  state  1  to  state  0  when  the  machine  breaks  down.  The  length  of  time  spent  at 

each  visit  to  state  0  is  distributed  as  M(X),  and  the  length  of  time  spent  at  each 
visit  to  state  1  is  distributed  as  M(v);  so  this  birth  and  death  process  is  the 
two-state  Markov  process  just  described. 


The  forward  equations 


Since  the  two-state  Markov  process  is  a  birth  and  death  process,  the  differential- 
difference  equations  for  the  process  may  be  written  down  by  substituting  the 
parameters  given  in  (2.1)  into  the  Kolmogorov  forward  equations  (1.7)  for  a 
general  birth  and  death  process.  This  gives 


jtPo(t)  =  vpl{t)-Xp0(t\ 


(2.2a) 


jtPi(t)  =  Xp0(t)-vp1(t).  (2.2b) 

At  any  time  t,  the  process  must  be  in  either  state  0  or  state  1,  so  p0(t )  +  p^t)  =  1 
for  all  t  >  0.  Substituting  p^t)  =  1  -  p0(t)  in  Equation  (2.2a)  leads  to 

jtPo(t)  +  (X  +  v)p0(t)  =  v. 


This  is  similar  to  the  differential  equations  you  have  met  previously  for  the 
Poisson  process  and  the  simple  birth  process.  It  can  be  solved  directly  using  the 
integrating  factor  method.  Multiplying  by  e{X+v)t,  we  have 

{ltP  °(t))e'i+V)'  +  Po«U  +  v)eM+,)'  =  ve«+>* 


or 

~(po(t)ea  +  v)l)  =  veu  +  v)l, 
and  integrating  this  equation  gives 

p0(t)e^+'>'  =  ~e>x+'>'  +  c.  (2.3) 

To  determine  the  constant  c,  the  state  of  the  process  at  time  t  =  0  must  be 
specified.  Now  recall  that  at  the  end  of  the  last  section  the  notation  px(t)  was 
introduced  as  a  simplification  for  pl<x{t),  the  probability  that  the  process  is  in  state 
x  at  time  t,  given  that  it  is  in  state  i  at  time  t  =  0.  So  suppose,  for  example,  that 
the  machine  is  under  repair  when  it  is  first  observed,  so  that  the  process  is  initially 
in  state  0.  Then  px(t)  is  a  simplification  for  p0lX(t),  and  Equation  (2.3)  is  an 
equation  for  p0t 0(t).  Now  p0  o(0)  =  1  and  po>1(0)  =  0  so  putting  t  =  0  in  Equation 
(2.3),  we  have 


1  = 


+  c, 


X  -1-  v 

and  hence  c  =  X/(X  +  v).  So 


Po.oW  = 


+ 


X 


,-(A  +  v)J 


X  +  v  X  +  v 
and,  since  po,i(0  =  1  —  Po,o(4  we  have 


Po,i(0  — 


A  +  V 


(1 


_  «-(A  +  t;)r 


)• 


(2.4a) 

(2.4b) 


Question  2.2  Solve  the  differential  equations  (2.2)  given  that  the  process  is  initially 
in  state  1.  □ 

Question  2.3  For  a  two-state  Markov  process  with  X  =  1,  v  =  calculate  the 
probabilities  that  the  process  is  in  state  0  at  times  t  =  0.2,  0.5,  1,  2,  4,  given  that 
(a)  it  starts  in  state  0  and  (b)  it  starts  in  state  1.  □ 


2.2  The  equilibrium  distribution 

To  appreciate  the  behaviour  of  the  random  process  it  is  helpful  to  calculate  some 
numerical  values.  From  Solution  2.3,  you  can  see  that  when  t  =  0  2  the 
probabilities  of  being  in  state  0  are  very  different  for  the  two  initial  states;  in  this 
short  time  interval,  the  process  is  unlikely  to  have  changed  states.  However,  by 
time  t  —  4  the  probabilities  are  very  close,  and  both  are  approximately  equal  to  j; 
the  initial  state  does  not  appear  to  have  very  much  effect.  We  shall  look  more 
closely  at  why  this  is  so. 
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When  the  initial  state  is  0,  the  probability  of  being  in  state  0  at  time  t  is  given  by 


Po.oW  = 


+ 


X  +  V  X  +  V 


e~(x+ v)f 


As  t  -*■  co,  e  <A  +  V)'  ->•  0  very  rapidly;  in  fact,  when  (a  +  v)t  =  5,  e~(X+v)t  is 
practically  negligible.  Hence  po>o(0  -  v/(X  +  v)  and  p0>1{t)  X/(X  +  v)  as  t  ->  oo,  and 
these  limiting  probabilities  are  approached  quite  rapidly. 


Question  2.4  Calculate  the  limiting  probabilities  for  a  two-state  Markov  process 
that  is  initially  in  state  1.  □ 


The  solution  to  this  question  shows  that  whether  the  process  starts  in  state  0  or  in 
state  1,  px(t),  the  probability  that  the  process  is  in  state  x  at  time  t,  approaches  the 
same  limit  as  t  -*■  co.  This  limit  is  denoted  by  px,  so  we  have 


P  o  = 


v 

I+C 


Pi  = 


X 

X  +  v’ 


Since  p0  +  pi  =  1,  {Px}  is  a  probability  distribution— the  limiting  distribution  of 
the  process.  You  might  have  expected  that  such  a  distribution  would  exist  for,  after 
a  long  time,  the  process  will  have  switched  many  times  between  states,  and  so  its 
starting  point  is  irrelevant.  The  example  in  Question  2.3  illustrates  this.  In  that 
example,  2=1  and  v  =  J  so  p0  =  J;  and  the  values  of  p0i0(t)  and  p1>0(t)  that  you 
calculated  for  t  =  4  are  both  very  close  to  y.  Thus,  by  time  t  =  4,  the  initial  state 
has  little  effect  on  the  probability  of  being  in  state  0.  So  the  primary  interpretation 
of  the  limiting  probability,  px,  is  as  the  probability  that,  after  the  process  has  been 
running  for  a  long  time,  it  will  be  in  state  x.  For  the  machine  example,  p0  and  px 
are  the  probabilities  that,  after  a  long  time  has  elapsed,  the  machine  will  be  under 
repair,  and  working,  respectively. 


Another  interpretation  of  the  limiting  probability  px  is  as  the  expected  proportion 
of  time  spent  in  state  x  over  a  long  period.  So,  for  the  machine,  p0  is  the  expected 
proportion  of  time  that  the  machine  is  under  repair,  and  pt  is  the  expected 
proportion  of  time  that  it  is  working.  The  limiting  distribution  of  a  two-state 
Markov  process  is  the  continuous-time  analogue  of  the  limiting  distribution  for  a 
Markov  chain  that  was  studied  in  Unit  6. 


So  far  we  have  considered  what  happens  when  the  process  is  known  to  start  in 
state  0  or  in  state  1.  However,  sometimes  the  initial  state  is  specified  in  the  form  of 
a  probability  distribution;  this  situation  is  considered  in  the  next  question. 


Question  2.5  Suppose  that,  for  i  =  0, 1,  the  probability  that  a  two-state  Markov 
process  is  in  state  i  at  time  0  is  ah  where  a0  +  ax  =  1. 

(i)  Express  px{t),  the  probability  that  the  process  is  in  state  x  at  time  r.  in  terms  of 

ao >  ai,  Po,*(0  and  P i ,*(*)■  Does  the  process  have  a  limiting  distribution  for  all 
values  of  a0  and  a,? 

(ii)  If  a0  =  v/(X  +  v),  calculate  the  probability  that  at  time  t  the  process  is 

(a)  in  state  0,  and  (b)  in  state  1.  □ 

In  Solution  2.5(i),  we  have  shown  that  the  two-state  Markov  process  always  has  a 
limiting  distribution  and  that  it  is  equal  to  {px}  for  any  probability  distribution  of 
the  initial  state.  In  Solution  2.5(ii),  where  the  initial  distribution  {ax}  is  taken  to  be 
the  limiting  distribution  {px},  we  found  that  px(t)  =  px  for  all  t.  So  the  limiting 

distribution  is  also  a  stationary  distribution  for  the  process.  In  all  the  Markov  This  is  the  continuous-time  analogue 

processes  in  continuous  time  that  we  study  in  this  course,  the  stationary  °f  a  stationary  distribution  for  a 

distribution  is  the  same  as  the  limiting  distribution;  and  so,  without  ambiguity,  Markov  chain, 
they  may  both  be  referred  to  as  the  equilibrium  distribution. 

It  is  important  to  realize  the  difference  between  a  stochastic  equilibrium  and  a 
mechanical  equilibrium.  The  two-state  Markov  process  changes  state  from  1  to  0 
and  back  again  according  to  given  criteria;  and  changes  continue  to  occur  at 
random  times,  so  the  process  is  not  in  equilibrium  in  any  mechanical  sense. 

However,  for  large  t,  at  any  instant  the  probability  of  the  process  being  in  any 
given  state  has  a  fixed  value,  independent  of  time. 
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There  is  another  way  in  which  the  equilibrium  distribution  can  be  obtained.  This 
method  will  be  used  for  other  processes  later  in  the  course;  for  example  in 
Section  5  of  this  unit,  and  in  Unit  9.  So,  as  it  is  often  the  easiest  way  of  obtaining 
the  equilibrium  distribution,  the  method  will  be  demonstrated  for  the  two-state 
Markov  process. 

If  it  is  assumed  that  an  equilibrium  distribution  { px }  exists,  then  it  must  satisfy  the 
differential  equations  (2.2).  Also,  by  definition,  an  equilibrium  distribution  is 

independent  of  time,  so  ^  =  0  and  ^-  =  0.  So,  substituting  in  the  forward 

equations  (2.2),  and  noting  that  px(t)  -*  px  as  t  ->  oo,  we  have 

0  =  vpi  —  Ap0, 

0  =  Ap0  -  v  Pi . 

These  equations  are,  of  course,  equivalent.  Since  {px}  is  a  probability  distribution, 
Po  +  Pi  =  and  so 

vpi  =  Ap0  =  A(1  -  p^, 
and  hence,  as  before,  we  have 

A  v 

Pi  —  i  .  >  Po  —  ;  • 

A  +  v  A  +  v 

So  the  equilibrium  distribution  can  also  be  obtained  by  setting  the  derivatives 
JtPx(t)  equal  to  zero  in  the  Kolmogorov  forward  equations,  and  then  solving  the 
equations  thus  obtained. 

The  main  results  of  this  subsection  on  the  equilibrium  distribution  for  a  two-state 
Markov  process  are  summarized  below. 


(i)  For  a  two-state  Markov  process,  px(t),  the  probability  that  the  process  is 
in  state  x  at  time  t,  tends  to  a  limit  px  as  t  ->  oo,  this  limit  being  the 
same  whatever  the  distribution  of  the  initial  state  of  the  process.  {px}  is  a 
probability  distribution,  the  limiting  distribution  of  the  process. 

(ii)  If  the  initial  state  has  the  probability  distribution  {px},  then  px(t)  =  px  for 
all  t;  so  the  limiting  distribution  is  also  a  stationary  distribution  of  the 
process.  This  distribution  is  known  as  the  equilibrium  distribution. 


(iii)  The  equilibrium  distribution  can  be  obtained  by  setting  ^px(r)  equal  to 

zero  in  the  Kolmogorov  forward  equations,  and  solving  the  linear 
equations  thus  obtained. 


We  close  this  section  by  considering  an  extension  of  the  model  to  a  situation  in 
which  a  number  of  two-state  processes  occur  simultaneously.  For  instance,  we 
might  consider  n  machines  which  break  down  from  time  to  time  and  where  each 
breakdown  is  mended  immediately;  or  n  telephone  lines  each  of  which  may  be  in 
use  or  free;  or  a  herd  of  n  cows  each  of  which  is  either  standing  and  grazing  or 
sitting  and  chewing  the  cud.  In  each  case  we  assume  that  the  n  objects  act 
independently  of  each  other. 

Example  2.1 

Suppose  there  is  a  herd  of  n  cows,  each  of  which  stands  and  grazes  (state  1)  for  a 
time  which  is  distributed  as  M(v),  and  then  sits  and  chews  the  cud  (state  0)  for  a 
time  distributed  as  M(A).  If  the  cows  are  all  standing  at  time  t  =  0,  what  is  the 
distribution  of  the  number  standing  at  time  t,  and  what  is  the  limiting  distribution 
of  the  number  standing?  What  is  the  expected  number  of  cows  standing  after  a 
long  time? 
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Solution 


Each  of  the  n  cows,  independently  of  the  others,  is  standing  at  time  t  with 
probability  p1A(t)  where,  from  Solution  2.2, 


P  i,i(0  = 


+ 


2  +  v  X  +  v 


g-(A  +  v)r 


So  the  number  of  cows  standing  at  time  t  has  a  binomial  distribution,  B(n,p1 

After  a  long  time,  the  probability  that  any  particular  cow  is  standing  is  given  by 
the  limiting  probability  =  2/(2  +  v),  so  the  number  of  cows  standing  has  a 
binomial  distribution  B(n,  2/(2  +  v)).  This  is  the  limiting  distribution  of  the  number 
of  cows  standing. 


The  expected  number  of  cows  standing  after  a  long  time  is  equal  to  the  mean  of 
this  limiting  distribution,  and  this  is  nX/(X  +  v).  □ 


Question  2.6  Suppose  that,  at  time  t  =  0,  a  of  the  cows  in  Example  2.1  are 
standing,  and  n  —  a  are  sitting  and  chewing  the  cud.  Find  the  expected  number  of 
cows  standing  at  time  t.  □ 


3  The  simple  birth-death  process 


In  Unit  7  the  simple  birth  process  and  the  pure  death  process  were  analysed.  In 
this  section,  the  concepts  of  these  two  models  are  combined  to  give  a  model 
known  as  the  simple  birth-death  process.  The  model  is  particularly  appropriate  to 
the  development  of  a  colony  of  simple  organisms  such  as  bacteria  or  yeasts;  and  it 
is  fairly  realistic,  at  least  in  the  early  stages  of  development. 

As  in  the  simple  birth  process,  we  assume  that  each  individual,  independently  of  all 
other  individuals,  gives  birth  to  new  individuals,  one  at  a  time,  with  rate  fi.  So  in 
any  time  interval  [t,t  +  8t]>  each  individual  has  probability  pdt  +  o(St)  of  giving 
birth  to  a  new  individual.  This  probability  is  independent  of  time  and  also  of  the 
age  of  the  individual;  so  the  model  is  clearly  not  suitable  for  higher  forms  of  life. 
Since  all  individuals  act  independently,  if  the  size  of  the  population  is  x  at  time  t, 
the  probability  of  one  individual  being  born  in  the  interval  [t,t  +  <5t]  is 
flxdt  +  o(St).  Hence,  as  for  the  simple  birth  process,  fix,  the  overall  birth  rate  of 
the  process,  is  equal  to  fix. 

Independently  of  giving  birth,  each  individual  in  the  population  is  liable  to  die.  As 
in  the  pure  death  process,  the  lifetime  of  an  individual  has  an  exponential 
distribution  with  parameter  v,  and  the  probability  that  an  individual  dies  in  the 
time  interval  [t,t  +  St]  is  equal  to  vdt  +  o(St).  Thus  if  the  population  size  is  x  at 
time  t,  the  probability  that  a  death  occurs  in  [t,t  +  <5t]  is  vxSt  +  o(5t),  and  v„  the 
overall  death  rate,  is  equal  to  vx.  So  this  model  is  a  birth  and  death  process  with 
overall  birth  and  death  rates  px  =  fix  and  vx  =  vx. 

In  Subsection  3.1  we  shall  begin  by  finding  the  p.g.f.  of  X(t),  the  number  of 
individuals  alive  at  time  t  in  a  simple  birth-death  process.  Then,  in  Subsection  3.2, 
we  shall  use  the  p.g.f.  to  derive  several  properties  of  the  process. 


3.1  The  probability  generating  function  of  the  simple  birth-death  process 

In  this  subsection,  we  calculate  the  p.g.f.  of  X(t),  the  number  of  individuals  alive  at 
time  t.  The  first  step  is  to  write  down  the  differential-difference  equations  for  the 
simple  birth-death  process;  these  can  be  obtained  by  substituting  px  =  px,  vx  =  vx 
in  the  Kolmogorov  forward  equations  (1.7).  This  gives 

jtpx(t)  =  P(x  -  1  )px-1(t)  +  v(x  +  1  )px+l(t)  ~(p  +  v)xpx(t),  x  =  1,2,  ..., 

jtPo(t)  =  vpM 


13 


Question  3.1  The  p.g.f.  of  X(t),  the  number  alive  at  time  t,  is 

co 

n (s,t)  =  £  px(t)sx. 

x  =  0 

By  multiplying  the  forward  equation  in  (3.1)  involving  — px(t)  by  sx  and  summing 
over  x,  obtain  a  partial  differential  equation  satisfied  by  fl(s,  r).  □ 


The  solution  to  this  question  shows  that  Il(s,r),  the  p.g.f.  of  the  number  alive  at 
time  t,  satisfies  the  partial  differential  equation 

dn  (02 

—  =  (/ls2  +  v-(P  +  v)s)~ .  (3.2) 

This  equation  is  linear  in  and  so  it  is  an  example  of  Lagrange’s  equation 

and  can  be  solved  using  the  procedure  described  in  Section  3  of  Unit  7.  The  next 
three  questions  are  based  on  the  steps  involved  in  applying  this  procedure  to 
Equation  (3.2),  and  will  give  you  practice  at  solving  partial  differential  equations 
by  this  method. 


Question  3.2 

(i)  Step  1:  Rewrite  Equation  (3.2)  in  Lagrange  form, 


jn  du  , 

and  identify  the  functions  /,  g  and  h. 

(ii)  Step  2:  Write  down  the  auxiliary  equations. 


□ 


The  third  step  involves  finding  two  solutions  to  the  auxiliary  equations.  To  do  this 
you  will  need  to  factorize  v  -  (p  +  v)s  +  ps2  into  (v  -  ps){  1  -  s).  And  for  /?  #  v, 
you  will  need  to  use  the  following  result: 

_ =1 _ _ L.) 

(v  —  —  s)  P  —  V\\  —  S  V  —  psJ 

We  shall  complete  the  solution  first  for  p  #  v,  and  find  the  solution  for  equal  birth 
and  death  rates  afterwards. 


Question  3.3 

(i)  Step  3:  For  p  #  v,  solve  the  auxiliary  equations,  writing  your  two  solutions  in 
the  form 

Ci  =  (f> i(s,  t,  LI),  c2  =  (j)2(s,  t,  ri). 

(ii)  Step  4:  Obtain  the  general  solution  of  the  partial  differential  equation.  □ 

The  final  step  is  to  find  the  particular  solution  for  a  given  initial  population  size. 
This  is  done  by  using  the  initial  condition  to  identify  the  arbitrary  function  in  the 
general  solution. 


Question  3.4  Step  5 

(i)  For  p  #  v,  find  the  p.g.f.  n(s,t)  given  that  there  is  a  single  individual  in  the 
population  at  time  t  =  0. 

(ii)  Show  that,  when  there  are  x0  individuals  in  the  population  at  time  t  =  0,  the 
p.g.f.  is  given  by 

'v(l  -  s)  -  (v  -  Ps)e(v~p)t Y° 


n  (s,  t)  = 


P{  1  -  s)  -  (v  -  Ps)e{v~p)t 


□ 


(3.3) 


This  completes  the  solution  for  the  case  P  ^  v,  when  the  birth  and  death  rates  are 
unequal.  The  solution  is  very  similar  for  equal  birth  and  death  rates,  p  =  v,  so  the 
details  are  left  for  you  to  complete  in  the  next  question. 
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Question  3.5  When  /?  =  v,  the  partial  differential  equation  (3.2)  for  TT(s,  t ) 
simplifies  to 


(i)  Show  that  the  general  solution  of  this  equation  is  given  by 


n  (s,  t)  =  4 > 


fit  + 


(3.4) 


where  ijy  is  an  arbitrary  function. 

(ii)  Tf  there  are  x0  individuals  in  the  population  at  time  t  =  0,  show  that  the  p.g.f. 
II(s,  t)  is  given  by 


n  M)  = 


( fit  —  sfit  +  sY° 
\fit  —  sfit  +  \) 


□ 


(3.5) 


The  p.g.f.  contains  full  information  on  the  probability  distribution  of  X(t);  in  the 
next  subsection  it  is  used  to  obtain  some  basic  properties  of  the  simple  birth-death 
process. 


3.2  Some  properties  of  the  process 

Some  properties  of  the  simple  birth-death  process  are  apparent  from  the  model. 
For  example,  since  both  births  and  deaths  can  occur,  the  size  of  the  population 
may  fluctuate.  This  is  different  from  the  simple  birth  process,  where  the  size  of  the 
population  can  only  increase,  and  the  pure  death  process,  where  the  size  only 
decreases  and  eventually  reaches  zero.  In  the  simple  birth-death  process,  the  size 
of  the  population  may  decrease  to  zero;  and  once  there  it  remains  there— if  no 
individuals  are  present  then  no  births  can  occur.  But  is  extinction  certain  to  occur? 
Or  is  it  possible  for  the  population  size  to  fluctuate  indefinitely  and  remain  finite? 
Or  will  the  population  grow  without  limit?  By  looking  at  the  p.g.f.  we  can  answer 
questions  such  as  these;  we  can  also  derive  the  distribution  of  X(t),  and  find  its 
mean  and  variance.  We  shall  begin  by  discussing  the  behaviour  of  the  process  after 
it  has  been  running  for  a  long  time,  and  in  particular  we  shall  find  the  probability 
that  extinction  will  occur. 


Extinction 


Since,  by  definition, 

H (s,t)  =  £  px(t)sx, 

x  =  0 

setting  s  =  0  gives 

n(0,  t)  =  Po(t)  =  p(x(t)  =  o). 

So  the  probability  that  no  individual  is  alive  at  time  t,  and  hence  that  extinction 
has  occurred  by  this  time,  is  given  by  11(0,0-  Setting  s  =  0  in  Expression  (3.3),  we 
obtain,  for  fi  #  v, 


n(0, 0  = 


v  _  ve<*-«*V o 
.fi  -  ve(v~P)tJ  ' 


(3-6) 


One  of  the  interesting  properties  of  any  Markov  process  is  its  behaviour  after  a 
long  time  has  elapsed,  that  is,  as  t  -»■  oo.  Now,  as  t  -*■  oo,  e(v_/?)'  ->  0  when  /?  >  v, 
and  e(v-/?)'  -»  co  when  /?  <  v;  so  letting  t  ->  oo  in  Expression  (3.6),  we  have 

lim  n(0, 0  =  I  G)  When  ^  >  V  (3.7) 

[  1  when  ft  <  v. 

So  ultimate  extinction  is  certain  if  the  death  rate  is  greater  than  the  birth  rate. 
However,  if  the  birth  rate  is  greater  than  the  death  rate,  the  probability  of  ultimate 
extinction  is  ( v/(3)x° ,  and  there  is  a  positive  probability  that  the  population  does  not 
die  out.  So  if  it  does  not  die  out,  does  it  remain  finite  in  size  or  increase  without 
limit? 


To  investigate  this,  we  let  t  -> 


lim  n(s,  t )  = 

t~>  cc 


co  in  Expression  (3.3)  for  IT(s,  t),  giving 
when  /?  >  v 
when  P  <  v. 


So  if  p  <  v,  the  limiting  value  of  fl(s,  t)  is  1  for  all  s,  and  this  limit  is  a  p.g.f.  This 
confirms  that  the  population  size  is  certain  to  be  zero  eventually  and  that  the 
population  will  die  out.  If  p  >  v,  the  limiting  value  of  fl(s,  t)  is  (v/p)x°.  Since  this  is 
less  than  1  when  s  =  1,  this  limit  is  not  a  proper  p.g.f.  There  is  a  positive 
probability  that  the  population  does  not  become  extinct,  but  the  probability  of  any 
finite  size  is  zero;  and  so,  if  extinction  does  not  occur,  the  population  size  increases 
without  bound.  This  result  is  similar  to  that  for  the  generation  size  in  a  branching 
process  after  a  large  number  of  generations. 


Unit  4,  Section  4 


Question  3.6  Suppose  that  a  simple  birth-death  process  has  equal  birth  and 
death  rates. 

(i)  Find  the  probability  that  extinction  will  have  occurred  by  time  t. 

(ii)  Find  the  probability  of  ultimate  extinction.  □ 


The  results  that  we  have  obtained  on  the  behaviour  of  a  simple  birth-death 
process  are  summarized  below. 


For  a  simple  birth-death  process  with  birth  rate  p  and  death  rate  v, 

(i)  when  p  <  v,  extinction  is  certain; 

(ii)  when  p  >  v,  the  probability  of  extinction  is  ( v/P)x° ,  where  x0  is  the  initial 
population  size. 


The  distribution  of  the  population  size 


When  the  process  starts  with  a  single  individual  and  /i  #  v,  the  p.g.f.  can  be 
rewritten  as 


n  (s,  t )  = 


v(l  —  e 


P  —  ve 


This  is  of  the  form 


(v -/?>')  _  S(v  _  pe^-P)') 
(v~p)t  -  ps(  1  -  e(v~p)t) ' 


H(s)  = 


a  —  bs 
c  —  ds' 


where  a  =  v(l  -  p),  b  =  v  -  pp,  c  =  p  -  vp,  d  =  p{  1  -  p)  and  p  =  e{v~p)t. 


Expanding  the  p.g.f.  IT(s)  =  {a  —  bs)/(c  —  ds)  as  a  series  of  powers  of  s  using  the 
Binomial  Theorem,  we  have,  for  d  <  c, 


n  (s)  = 


a  —  bs 
c  —  ds 


(a  ~  bs)  ^  _  ds^j  1 


(a-bs)^ 


a 

c 


ad  —  be 


So  for  x  =  1,2,  ...,  the  probability  distribution  has  a  geometric  form,  though  the 
term  for  x  =  0  does  not  follow  the  same  pattern.  This  is  a  modified  geometric 
distribution. 


If  p  >  v,  then  p  =  e{v  p)t  <  1,  so  d/c  =  p(  1  -  p)/(p  -  vp)  <  1 ;  and  if  p  <  v,  then 
P  >  1>  ar,d  so  d/c  =  P(p  —  1  )/(vp  —  P)  <  1.  So  for  p  >  v  and  for  p  <  v,  the  binomial 
expansion  is  valid. 


Basic  results  for  the  modified 
geometric  distribution  are  given  in 
the  Handbook.  You  should  be  able 
to  recognize  the  p.g.f.  of  this 
distribution,  and  apply  these  results. 
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Substituting  the  values  of  a,  b,  c  and  d,  and  noting  that  ad  -  be  =  p(v  -  B)2  where 
p  =  e(v  P)t,  we  have 


V(1  ~ P ) 
P-vp’ 


Px(t)  = 


(d.y'ad  -be  _  (/?(!  -  p)Y^p(v  - 
W  c2  (£-V/>)*+1 


*  =  1,2,  ...  . 


The  exact  terms  of  this  distribution  are  not  important;  but  the  general  modified 
geometric  form  is  worth  noting.  Because  of  this  form,  the  probability  that  the  size 
is  x  decreases  as  x  increases:  large  population  sizes  are  less  likely  than  small  ones. 


As  t  ->  co,  p  —  e(v  P)t  0  when  /?  >  v  and  p  ->  oo  when  ft  <  v.  So,  as  t  oo 
Px(t)  0  for  x  =  1,2,...,  and 


Po(t) 


P 


when  (1  >  v 


[  1  when  /I  <  v. 

This  is  Result  (3.7)  for  the  special  case  x0  =  1. 


The  calculation  of  the  mean  and  variance  of  X(t)  for  a  simple  birth-death  process 
starting  with  one  individual  is  simplified  by  using  the  p.g.f.  II(s)  =  (a  -  bs)/(c  -  ds), 
as  is  shown  in  the  following  question. 


Question  3.7 

(i)  Calculate  the  mean  and  variance  of  a  random  variable  with  p.g.f. 

El(s)  =  (a  —  bs)/(c  —  ds). 

(ii)  Hence  determine  the  mean  size  at  time  t  and  the  variance  for  a  population 
starting  from  one  individual  and  developing  according  to  a  simple  birth-death 
process.  □ 


Returning  to  the  p.g.f.  at  time  t, 


n(s,  t)  = 


( v(l  -  s)  -  (v  -  jfoy»-wY° 
V/?(l  -  s)  -  (v  -  psW'-n) 


you  can  see  that  it  is  the  product  of  x0  identical  terms,  each  of  which  is  the  p.g.f. 
for  a  simple  birth-death  process  starting  with  one  individual.  So  X(t )  can  be 
thought  of  as  the  sum  of  x0  independent  variates,  each  of  which  denotes  the  size  at 
time  t  of  a  simple  birth-death  process  starting  with  one  individual.  In  other  words, 
there  are  x0  processes  developing  independently  of  each  other.  This  is  a  direct 
result  of  one  of  the  basic  assumptions  of  the  model,  that  individuals  give  birth  and 
die  independently  of  each  other.  Use  this  result  to  answer  the  next  question. 


Question  3.8  Find  the  mean  population  size  at  time  t  and  the  variance  for  a 
simple  birth-death  process  starting  with  x0  individuals  at  time  t  =  0.  □ 


As  in  the  case  /?  #  v,  when  the  birth  and  death  rates  are  equal  the  distribution  of 
the  size  of  the  population  at  time  t  can  be  thought  of  as  the  sum  of  x0  variates, 
each  originating  from  a  single  individual.  The  p.g.f.  of  the  distribution  of  X(t)  when 
the  population  originates  with  a  single  individual  is 


n(s,  t)  - 


(k  -  s(pt  -  1) 
fit  +  1  —  Spt  ' 


This  is  again  of  the  form  (a  -  bs)/(c  -  ds);  and  by  a  similar  argument  to  that  used 
for  the  case  (1  #  v,  we  can  show  that  the  distribution  at  time  t  has  a  modified 
geometric  distribution,  and  is  given  by 


Po(0  = 


Pt 

1  +  fit* 


Px{t)  = 


(■ pt)x- 1 

(1  +pt)x+1 


x  =  1,2,  ...  . 
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Question  3.9  Suppose  that  a  simple  birth-death  process  has  equal  birth  and 
death  rates. 

(i)  Use  the  results  from  Question  3.7(i)  to  calculate  the  mean  and  variance  of  the 
population  size  at  time  t,  if  the  process  starts  with  one  individual  at  time  t  =  0. 

(ii)  Write  down  the  mean  and  variance  of  the  population  size  at  time  t,  if  the 

process  starts  with  x0  individuals  at  time  t  =  0.  □ 

The  deterministic  model 

In  Section  4  of  Unit  7  you  saw  how  to  obtain  the  deterministic  model 
corresponding  to  a  stochastic  process  in  which  the  only  possible  changes  in  the 
population  size  are  an  increase  of  one  or  a  decrease  of  one.  In  the  final  question  of 
this  section,  this  method  is  required  to  derive  the  deterministic  model  for  the 
simple  birth-death  process. 

Question  3.10 

(i)  Obtain  the  differential  equation  for  the  deterministic  model  of  the  simple 
birth-death  process. 

(ii)  Show  that  for  a  population  whose  size  is  z0  at  time  t  =  0,  the  population  size 
at  time  t  is  given  by 

2  =  z0e(P~v)'. 

(iii)  Briefly  compare  the  behaviour  of  the  stochastic  model  and  that  of  the 
deterministic  model  for  the  simple  birth-death  process.  □ 

In  this  section  we  have  derived  the  basic  properties  of  the  simple  birth-death 
process.  In  the  next  section  we  shall  study  some  further  properties  and  also  relate 
the  process  both  to  a  random  walk  and  to  a  branching  process. 


4  More  about  the  simple  birth-death  process 


In  this  subsection  we  shall  obtain  some  further  results  for  the  simple  birth-death 
process.  First,  in  Subsection  4.1,  we  shall  ignore  the  times  at  which  events  occur 
and  consider  properties  of  the  process  which  are  independent  of  time.  Then,  in 
Subsection  4.2,  we  shall  study  the  times  to  various  events. 


4.1  Embedded  processes 

A  random  walk 

Suppose  that  instead  of  observing  the  complete  birth-death  process,  the  process  is 
observed  only  at  those  instants  when  a  birth  or  a  death  actually  occurs.  These  are 
the  points  of  time  at  which  there  is  a  discontinuity  in  X(f),  the  size  of  the 
population;  at  each  of  these  points  the  size  changes  by  one  unit.  We  can  relabel 
the  instants  of  time  when  changes  occur  as  n  =  0, 1,2,3, ...,  and  think  of  n  as  a 
new  discrete-time  variable.  This  leads  to  a  new  random  process  {Xn;  n  =  0, 1,  ...} 
in  discrete  time.  The  new  process  is  said  to  be  embedded  in  the  original  one,  and  it 
is  referred  to  as  an  embedded  process.  The  random  variable  Xn  is  the  size  of  the 
embedded  process  at  time  n,  and  is  equal  to  the  size  of  the  population  immediately 
after  the  instant  of  the  nth  change. 

Figure  4.1  shows  a  realization  of  a  simple  birth-death  process  starting  with  three 
individuals.  And  Figure  4.2  shows  the  same  realization  as  an  embedded  process;  in 
the  figure,  X0  =  3,  Xt  =  4,  X2  =  3, . . . . 
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Figure  4.1  A  realization  of  a  simple  birth-death  process 
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Figure  4.2  A  realization  of  the  embedded  process 

The  embedded  process  has  many  of  the  properties  of  the  original  process.  The 
value  of  Xn  can  only  increase  or  decrease  by  one  at  each  change,  the  probability 
that  the  next  change  is  an  increase  (or  a  decrease)  is  the  same  as  for  the  simple 
birth-death  process,  the  population  will  become  extinct  or  its  size  will  increase 
indefinitely,  and  it  will  attain  the  same  maximum  or  minimum  size.  However, 
results  involving  real  time  for  the  original  process  are  not  relevant  to  the 
embedded  process. 

In  order  to  study  the  embedded  process,  we  require  the  probabilities  that  at  any 
time  the  next  event  in  a  simple  birth-death  process  is  a  birth  or  a  death.  Suppose 
the  population  contains  x  individuals  at  time  t.  Then,  in  the  interval  [ t ,  t  4-  St],  the 
probability  that  a  birth  occurs  is  PxSt  +  o(St),  and  the  probability  that  a  death 
occurs  is  vxSt  +  o(5t).  Hence  the  probability  that  an  event  occurs  is 
(/?  +  v)x  St  +  o(St).  So  if  an  event  occurs,  the  conditional  probability  that  it  is  a 
birth  is 

P{a  birth  occurs  in  [t,  t  +  St]  |  an  event  occurs  in  [ t ,  t  +  <5t]) 

P( a  birth  occurs  in  [f,  t  +  &]) 

P(an  event  occurs  in  [r,  t  +  <5f]) 

_  fix  St  +  o(<5f) 

(/?  +  v)x  St  +  o(St) 

P  +  o{St)/St 

~  P  +  v  +  o{St)/St 

as  St  — *  0. 

P  +  v 

So  if  an  event  occurs,  the  probability  that  it  is  a  birth  is  p/(p  +  v)  and  the 
probability  that  it  is  a  death  is  1  -  p/(p  +  v)  =  v/(P  +  v).  These  probabilities  are 
independent  of  the  time  t  at  which  the  event  occurs  and  the  size  of  the  population 
at  that  time. 


By  definition,  o(5t)/St  -*•  0  as  St  -*  0. 


In  M245  Unit  8,  this  result  is 
derived  by  considering  a  pooled 
Poisson  process. 
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We  now  return  to  the  embedded  process  {Xn;  n  =  0, 1, . . .}.  If  the  simple  birth- 
death  process  starts  with  x0  individuals,  then  X0  =  x0.  And  for  n  =  0, 1,  . . . ,  we 
have 


P(Xn+  1  —  x  +  1  |  X„  =  x)  =  X  -  1,  2,  .  .  . , 

•  P(Xn+l  =  X  -  1  |A:„  =  x)  =  — x=l,2,  .... 

p  +  V 

Since  0  is  an  absorbing  state,  this  point  acts  as  an  absorbing  barrier  for  the 
process;  and  so  the  embedded  process  is  a  simple  random  walk  with  an  absorbing 
barrier  at  0,  in  which  p  =  p/(p  +  v)  and  q  =  v/(p  +  v).  So  the  results  that  were 
found  in  Section  2  of  Unit  5  for  gambling  against  a  casino  can  be  applied  to  the 
random  walk  embedded  in  a  simple  birth-death  process. 

In  Unit  5  we  found  that  if  Gary,  the  gambler,  starts  with  £j  and  plays  against  a 
casino,  then  qj,  his  probability  of  ruin,  is  given  by 

1  when  p  <  q 
({j  when  p  >  q, 

where  p  is  the  probability  that  he  wins  each  game,  q  is  the  probability  that  he  loses 
each  game,  and  the  stake  on  each  game  is  £1.  Setting  p  =  p/(p  +  v),  q  =  v/(y?  +  v) 
and  j  =  x0,  we  obtain  the  probability  that  the  random  walk  embedded  in  a  simple 
birth-death  process  is  eventually  absorbed  at  0;  and  this  is  the  probability  of 
ultimate  extinction  in  a  simple  birth-death  process  starting  with  x0  individuals. 
Thus  the  probability  of  ultimate  extinction  is  1  when  p  <  v  and  (v//?)*°  when  p  >  v. 
This  result  was  found  by  a  different  method  in  Section  3. 


This  result  was  obtained  in 
Frame  6A  of  the  audio-tape  session 
in  Unit  5. 


When  p  <  q,  the  expected  duration  of  Gary’s  match  is  j/(q  -  p);  and  when  p  =  q, 
the  expected  duration  is  infinite.  So,  for  a  simple  birth-death  process  starting  with 
x0  individuals,  the  mean  number  of  events  (births  and  deaths)  that  occur  before 
extinction  takes  place  is  x0(v  +  p)/(v  —  (1)  when  /i  <  v,  and  is  infinite  when  ft  =  v. 


Another  result  from  Unit  5  that  can  be  applied  to  a  simple  birth-death  process  is 
the  probability  of  ruin  in  the  classical  gambler’s  ruin,  in  which  Gary  and  his 
opponent  have  £a  between  them.  If  q}  is  Gary’s  probability  of  ruin  when  he  begins 
with  £/,  then  the  probability  that  he  eventually  wins  and  his  opponent  is  ruined  is 


1  -  ( q/p)j 

1  -  (q/pT 

j 


la 


when  p  ^  q 
when  p  =  q. 


This  can  be  used  to  find  the  probability  that  the  population  in  a  simple  birth- 
death  process  will  ever  reach  a  given  size.  Can  you  see  how  to  do  this?  Try  the 
next  question. 


This  result  was  obtained  at  the  end 
of  Section  2  in  Unit  5. 


This  result  was  obtained  in 
Frame  5A  of  the  audio-tape  session 
in  Unit  5. 


Question  4.1 

(i)  For  a  simple  birth-death  process  starting  with  x0  individuals,  find  the 
probability  that  the  size  will  ever  increase  to  k  individuals. 

(ii)  Find  the  probability  that  the  size  will  ever  increase  to  10  if  the  process  starts 
with  5  individuals,  when 

(a)  p  =  4,  v  =  6,  (b)  p  =  6,  v  =  4.  □ 

The  numerical  results  of  part  (ii)  of  this  question  are  quite  interesting.  When  p  =  4 
and  v  =  6,  extinction  is  certain;  and  in  this  case,  the  probability  that  the  size 
increases  to  ten  before  extinction  occurs  is  approximately  0.116.  On  the  other 
hand,  when  p  =  6  and  v  =  4,  the  probability  that  extinction  occurs,  given  five 
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individuals  initially,  is  only  (f)5  =  0.1317;  and  the  probability  that  the  size  ever 
increases  to  ten  is  0.8836.  So  for  this  process,  in  which  the  birth  rate  is  greater 
than  the  death  rate,  the  population  is  very  likely  to  double  its  original  size.  But 
suppose  that  we  know  this  population  does  in  fact  become  extinct;  in  this  case, 
what  is  the  probability  that  the  population  size  will  increase  to  ten  before 
extinction  occurs? 

We  shall  denote  the  event  that  extinction  occurs  by  E  and  the  event  that  the  size 
reaches  ten  by  T,  so 

P(E)  =  0.1317,  P(T)  =  0.8836. 

Now,  if  the  size  reaches  ten,  the  probability  that  extinction  occurs  is  then 
(v/j0)10  =  0.0173;  that  is, 

P(E  |  T)  =  0.0173. 

Hence,  by  Bayes’  Theorem, 

I  P^W) 

1  7  P(E) 

_  0.0173  x  0.8836 
0.1317 
=  0.116. 

That  is,  if  the  population  does  eventually  become  extinct,  it  is  not  very  likely  that 
its  size  will  ever  have  been  as  large  as  ten. 

A  branching  process 

One  discrete  random  process  embedded  in  a  simple  birth-death  process  is  a 
random  walk  with  an  absorbing  barrier;  another  is  a  Galton-Watson  branching 
process.  The  random  walk  is  based  on  the  size  of  the  population;  the  branching 
process  looks  at  the  individuals  separately. 

Each  individual  in  a  simple  birth-death  process  gives  birth  to  new  individuals, 
according  to  a  Poisson  process  with  rate  /?,  until  it  dies;  and  its  lifetime  has  an 
exponential  distribution,  M(v).  The  parameters  /?  and  v  are  the  same  for  all 
individuals.  So  each  individual  gives  birth  to  a  random  number  of  offspring, 
independently  of  all  other  individuals,  and  for  each  individual  the  number  of 
offspring  has  the  same  distribution.  These  are  precisely  the  conditions  for  a 
Galton-Watson  branching  process. 

Suppose  the  birth-death  process  starts  with  a  single  individual;  this  is  the  ancestor 
of  the  branching  process.  This  individual  gives  birth  to  a  random  number  of 
offspring,  who  form  the  first  generation,  and  then  dies.  These  offspring  behave 
identically,  each  giving  birth  to  a  random  number  of  individuals,  who  together 
form  the  second  generation,  and  then  dying.  So  the  generations  are  clearly  defined, 
although  the  offspring  in  a  generation  are  not  all  born  at  the  same  time. 

Some  properties  of  a  branching  process,  like  the  distribution  of  the  size  of  a 
particular  generation,  are  not  of  interest  in  the  birth-death  process  since  the 
members  of  a  generation  are  born  and  die  at  different  times  and  the  generations 
overlap.  But  properties  relating  to  extinction  are  relevant  to  the  birth-death 
process. 

In  order  to  obtain  explicit  results,  we  need  to  derive  the  distribution  of  X,  the 
number  of  offspring  born  to  an  individual.  Now  the  number  of  offspring  of  an 
individual  depends  on  T,  the  lifetime  of  the  individual,  whereas  we  want  to  find  the 
unconditional  distribution  of  A.  To  do  this,  we  shall  start  by  supposing  that  the 
lifetime  of  an  individual  has  a  fixed  value  t,  and  find  the  conditional  distribution  of 
X.  Then  we  shall  use  this  to  derive  the  unconditional  distribution. 


It  is  no  accident  that  this  probability 
is  equal  to  the  probability  obtained 
previously  that  the  size  will  increase 
to  ten  before  extinction  occurs  in  a 
simple  birth-death  process  with 
=  4,  v  =  6.  However,  you  need  not 
concern  yourself  here  with  the 
details  of  why  this  is  so. 
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Since  an  individual  gives  birth  according  to  a  Poisson  process  with  rate  /?,  the 
number  of  offspring  it  has  in  an  interval  of  length  t  has  a  Poisson  distribution  with 
parameter  pt,  so 

P(X  =  x\T=t)  =  —^t)X,  x  =  0,1, ... . 

Now  if  X  and  T  were  both  discrete  random  variables  we  could  use  the  Theorem  of 
Total  Probability  to  obtain  the  unconditional  distribution  of  X,  since  by  this 
theorem,  we  have,  for  any  discrete  random  variables  X  and  Y, 

P(X  =  x)  =  Y,P(X  =  x|  Y  =  y)P(Y  =  y). 
y 

However,  T  is  a  continuous  random  variable  with  p.d.f.  f{t)  =  ve~v,,t>  0,  so  we 
cannot  use  this  result  here.  So,  is  there  an  analogous  form  of  the  Theorem  of  Total 
Probability  that  we  can  use?  The  answer  to  this  is  yes;  the  form  of  the  theorem 
that  we  require  is  given  below. 


If  X  is  a  discrete  random  variable  and  Y  is  a  continuous  random  variable 
with  p.d.f.  g(y)  then,  for  all  xeQx, 

P(X  =  x)  =  I  P(X  =  x\Y=  y)g(y)  dy. 

J-ao 


Applying  this  result  to  our  problem,  we  have,  for  x  =  0, 1,  ..., 

pec 

P(X  =  x)=  P(X  =  x\T=  t)  ve~vl  dt 


e-p,(pty 


x! 


ve  vtdt 


vp 

x! 


txe~(p+v),dt. 


This  integral  can  be  calculated  using  the  following  formula,  which  may  be 
obtained  from  the  fact  that  the  p.d.f.  of  the  gamma  distribution  integrates  to  1 : 


y 


"  - 1  °~ay  dy  =  — - ^ 


Writing  n  =  x  +  1  and  a  =  /?  +  v  in  this  formula  leads  to 

x! 


P(X  =  x)  =  ^-x 


x!  ( p  +  v)x+1 


a 


p  +  V  \P  +  V 


,  x  =  0, 1,  .... 


So  the  number  of  offspring  of  an  individual  has  a  geometric  distribution,  G0(p), 
where  p  =  ft /(ft  +  v).  Its  mean,  p,  is  p/q  or  P/v,  and  its  p.g.f.  is 


n  (s)  = 


1  —  ps  P  +  V  -  Ps' 


Some  of  the  results  derived  in  Unit  4  for  branching  processes  can  now  be  applied 
to  a  simple  birth-death  process.  In  Section  4  of  that  unit  we  showed  that  the 
probability  of  ultimate  extinction  of  a  branching  process  starring  with  a  single 
individual  is  d*,  where  0*  is  the  smallest  positive  root  of  the  equation  9  =  n(0).  In 
Question  4.2  of  the  unit,  you  solved  this  equation  when  the  offspring  distribution 
is  G0(p),  and  found  9*  =  1  when  p  <  q  and  9*  =  q/p  when  p  >  q.  For  the  simple 
birth— death  process,  p  =  P/(P  +  v),  q  —  v/(p  +  v),  so  extinction  is  certain  when 
P  <  v  and  the  probability  of  extinction  is  v/p  when  p  >  v.  This  is  the  result  we 
obtained  in  Section  3,  for  the  case  x0  =  1. 


This  result  is  in  the  Handbook. 
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In  Subsection  5.1  of  Unit  4,  we  studied  a  Galton-Watson  branching  process  that 
started  with  x0  individuals.  We  found  that  if  p,  the  mean  number  of  offspring  of  an 
individual,  is  less  than  or  equal  to  1,  then  each  of  the  x0  branches  is  certain  to 
become  extinct,  whereas  if  //  >  1,  the  probability  of  extinction  of  all  branches  is 
(6*)x°.  Since  n  =  p/q  =  p/v,  this  leads  again  to  the  result  that  extinction  is  certain 
when  p  <  v,  and  the  probability  of  extinction  is  ( v/P)x°  when  p  >  v. 

In  Unit  4,  we  also  derived  results  about  the  expected  number  of  individuals  in  each 
generation  of  a  branching  process,  and  the  expected  total  number  of  individuals  in 
the  process.  The  expected  value  of  Z„,  the  number  of  individuals  in  generation  n,  is 
equal  to  //',  and  the  total  expected  number  of  individuals  who  ever  live  up  to  and 

including  the  nth  generation  is  E(Tn)  =  1  +  p  +  As  n  -*  oo,  E(Tn)  -*  oo  if  Unit  4,  Equation  (3.10) 

fi  >  1  and  E(Tn)  ->  1/(1  —  p)  if  p  <  1.  Unit  4,  Question  3.5 

For  a  simple  birth-death  process,  p  =  p/v.  Therefore  if  p  <  v,  so  that  extinction  is 
certain,  the  expected  total  number  of  individuals  to  live  in  the  population  is  equal 
to  1/(1  —  p/v)  or  v/(v  -  p)  if  p  <  v,  and  is  infinite  if  p  =  v.  So  if  p  =  v,  the 
population  is  certain  to  die  out  eventually  but  the  mean  number  of  individuals 
born  before  this  occurs  is  infinite.  When  p  <  v,  the  mean  depends  only  on  the  ratio 
P:v  or  equivalently  on  p  =  p/(p  +  v)  and  q  =  v/(p  +  v).  Table  4.1  shows,  for 
various  values  of  p  and  q,  the  mean  total  number  of  individuals  who  ever  live  in  a 
simple  birth-death  process  starting  with  one  individual. 


Table  4.1 


p 

0.1 

0.2 

0.3 

0.4 

0.45 

0.49 

0.5 

4 

0.9 

0.8 

0.7 

0.6 

0.55 

0.51 

0.5 

Mean  number  of  individuals 
that  ever  live 

— 

1.125 

1.333 

1.75 

3.0 

5.5 

25.5 

00 

You  can  see  that  the  mean  remains  small  until  p  is  as  large  as  0.4.  And  as  p 
approaches  0.5,  the  mean  increases  rapidly. 

Question  4.2  With  the  aid  of  branching  process  trees,  calculate  the  probability 
that  exactly  three  individuals  (including  the  initial  one)  will  exist  in  a  simple  birth- 
death  process  before  extinction  occurs.  □ 

In  this  subsection  we  have  looked  at  two  discrete-time  random  processes 
embedded  in  a  simple  birth-death  process.  Each  of  them  enables  us  to  obtain 
some  properties  of  the  simple  birth-death  process.  However,  since  they  are 
embedded,  they  tell  us  nothing  about  the  development  of  the  process  in  time.  In 
the  next  subsection,  some  properties  connected  with  time  will  be  derived. 


4.2  Times  to  various  events 

Most  of  the  results  obtained  so  far  for  the  simple  birth-death  process  have  been 
about  probabilities  of  various  events.  We  have  not  discussed  the  distributions  of 
the  times  until  these  events  take  place.  In  this  subsection  we  shall  derive  some 
simple  results  about  the  times  between  events,  and  the  time  until  extinction  occurs. 

If  a  population  contains  x  individuals  at  time  t,  then  the  probability  that  some 
event,  a  birth  or  a  death,  occurs  in  [r,  t  +  «5t]  is  (p  +  v)x  St  +  o{St).  Taken  together 
with  the  assumptions  about  multiple  events  and  independence  of  events,  this 
implies  that  the  time  to  the  next  event  has  an  exponential  distribution  with 
parameter  (p  +  v)x  and  mean  1  /(p  +  v)x.  So  the  larger  the  population  is,  the 
shorter  will  be  the  mean  time  between  events,  and  so,  as  one  would  guess,  the 
more  frequently  births  and  deaths  will  occur. 
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Another  result  that  can  be  obtained  concerns  the  time  to  extinction.  Now  the  time 
to  extinction  is  a  random  variable  T,  and 


P(T  <  t)  =  P(population  is  extinct  by  time  t) 

=  P(X(t)  =  0). 

In  Section  3,  we  found  this  probability  for  a  population  containing  x0  individuals 
.  initially.  Writing  F(t)  =  P(T<  t)  and  using  Result  (3.6),  we  have,  for  p  ^  v, 


F(t)  = 


v  —  ve 


,(v-p)t\x  0 


When  p  <  v,  extinction  is  certain;  and  since,  in  this  case,  e(v~p)l  -►  oc  as  t  -*  oo,  it 
follows  that  F(t)  ->  1  as  t  ->  co  and  T  has  a  proper  distribution.  When  ft  >  v, 
e<v~P)t  0  as  t  ->  go,  so  F{t)  ->  (v/p)x°;  and,  since  this  limit  is  less  than  1,  the  time  T 
is  said  to  have  an  improper  distribution.  As  you  have  already  seen,  this  limit  is  the 
probability  of  extinction.  The  form  taken  by  the  function  F(t)  is  illustrated  in 
Figure  4.3.  In  Figure  4.3(i),  p  =  2,  v  =  3  and  x0  =  2;  and  in  Figure  4.3(h),  p  =  3, 
v  =  2  and  x0  =  2. 


When  p  <  v,  F(t)  is  the  c.d.f.  of  T,  so  f{t),  the  p.d.f.  of  T,  may  be  obtained  by 
differentiating  F(t).  This  gives 


fit)  =  *0 


ve 


,(v-p)t  _  ,,Vo-i 


Vg<v-/m  _  p 


(v  —  P)2ve(v~p)l 


t  >0. 


(ve<v-/?)<  _  p) 2  ’ 

This  p.d.f.  has  a  rather  complicated  form.  Figure  4.4  shows  the  shape  of  its  graph 
for  various  values  of  P,  v,x0. 


When  x0  =  1,  the  c.d.f.  simplifies  to  give 
ve(v-/?><  _  v 


ve 


Xv-P)t 


-f 


t  >  0, 


The  signs  in  the  expressions  have 
been  reversed  to  make  each  term 
positive. 
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Question  4.3 

(i)  Find  the  function  F{t)  =  P(T  <  t),  where  T  is  the  time  to  extinction  in  a  simple 
birth-death  process  with  /?  =  v. 

(ii)  Show  that  T  has  a  proper  distribution,  and  calculate  its  p.d.f.  □ 

Question  4.4 

(i)  When  /?  =  v,  calculate  the  expected  time  to  extinction  for  a  population  starting 
with  one  individual. 

(ii)  What  can  you  say  about  the  expected  time  to  extinction  when  the  initial  size  is 

greater  than  1?  □ 
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5  The  effect  of  immigration 


In  Sections  3  and  4,  the  simple  birth-death  process  has  been  discussed  in  great 
detail.  This  process  models  a  population  which  is  closed  in  the  sense  that  the  only 
additions  are  births  to  existing  members — no  new  members  join  the  population  by 
immigration;  and  no  individual  leaves  except  by  dying.  However,  in  the 
immigration-birth  process,  which  was  studied  in  Unit  7,  the  population  grows  due 
to  immigration  as  well  as  births  to  existing  members — new  members  arrive 
according  to  a  Poisson  process. 

In  this  section  the  concepts  of  these  two  models  are  combined,  and  birth,  death 
and  immigration  are  all  considered.  Immigration  only  is  investigated  in  M245  as 
the  Poisson  process.  And  birth  only,  death  only,  and  birth  plus  immigration  were 
discussed  in  Unit  7.  In  Subsection  5.1,  we  shall  analyse  a  model  in  which  only 
immigration  and  death  are  permitted;  this  is  discussed  in  an  audio-tape  session. 

We  shall  then  have  considered  all  cases  where  only  one  or  two  of  immigration, 
birth  and  death  occur.  To  complete  our  discussion,  in  Subsection  5.2,  a  model  will 
be  considered  in  which  all  three  occur  together.  The  analysis  of  the  two  models 
discussed  in  this  section  does  not  involve  any  new  concepts;  the  ideas  and 
techniques  involved  are  those  introduced  in  Unit  7  and  earlier  sections  of  this  unit. 

5.1  The  immigration-death  model 

In  the  immigration-death  model,  individuals  arrive  and  join  the  population 
according  to  a  Poisson  process;  and  once  in  the  population  they  may  die  at  any 
time,  the  lifetime  of  each  individual  having  an  exponential  distribution. 

This  model  provides  a  fairly  realistic  representation  of  many  practical  situations. 
For  example,  the  population  might  consist  of  the  dust  particles  or  the  molecules  of 
a  gas  in  a  given  volume.  New  particles  may  enter  the  volume  at  any  time,  and  the 
Poisson  process  provides  a  reasonable  model  for  their  arrival  pattern.  Once  inside, 
any  particle  may  leave  at  any  time,  and  the  exponential  distribution  provides  a 
reasonable  model  for  the  time  spent  inside  the  volume. 

This  process  can  also  be  used  to  model  the  number  of  people  in  a  shop,  or  cars  in 
a  car  park,  provided  the  shop  or  car  park  is  large  enough  to  accommodate  all 
individuals  that  may  arrive.  And  it  arises  as  the  model  for  a  queueing  system  in 
which  customers  arrive  according  to  a  Poisson  process,  there  are  (theoretically) 
infinitely  many  servers,  and  the  service  time  of  each  customer  is  exponentially 
distributed.  The  theoretically  infinite  number  of  servers  means  that,  in  practice, 
there  are  sufficient  servers  for  each  customer  to  be  served  immediately  on  arrival. 
This  might  be  an  appropriate  model  for  a  telephone  system  with  sufficient  lines  to 
accommodate  all  callers,  and  where  the  ‘service’  is  the  conversation  and  it  can  be 
assumed  that  the  rate  of  incoming  calls  is  constant.  However,  in  practice,  the  rate 
would  probably  vary  over  a  period  of  several  hours,  so  the  model  would  apply 
only  for  the  period  during  which  the  rate  is  constant.  Other  queueing  models  are 
studied  in  the  next  unit. 

The  analysis  of  the  immigration-death  model  is  similar  to  the  analysis  of  the 
models  that  you  have  already  met;  it  is  developed  in  the  audio-tape  session.  But 
before  listening  to  the  tape,  think  for  a  while  about  the  behaviour  of  this  process. 

As  in  the  simple  birth-death  process,  the  population  size  will  fluctuate.  However, 
whereas  in  that  process  if  the  population  size  reaches  zero,  it  remains  zero,  in  this 
model  extinction  is  only  temporary,  for  eventually  a  new  immigrant  will  arrive.  So, 
since  permanent  extinction  cannot  occur,  will  the  population  eventually  increase 
without  bound?  Now  the  two-state  process,  discussed  in  Section  2,  can  be  thought 
of  as  a  simple  case  of  an  immigration— death  process.  Potential  immigrants  arrive 
according  to  a  Poisson  process,  but  only  actually  join  if  the  population  contains 
no  individuals.  Once  one  has  joined  no  more  can  join  until  that  one  has  died.  And 
you  saw  that  this  process  has  an  equilibrium  distribution  which  gives  the  expected 
proportion  of  time  spent  in  each  state  in  the  long  run.  Does  the  immigration- 
death  process  also  have  an  equilibrium  distribution?  That  is,  does  the  distribution 
of  the  population  size  at  time  t  approach  a  limiting  distribution  as  t  increases? 


26 


And  if  so,  does  this  happen  however  large  or  small  the  immigration  rate,  or  is  it 
possible  for  the  size  to  increase  indefinitely  if  the  immigration  rate  is  high  enough? 
These  questions  are  discussed  in  detail  on  the  audio-tape.  We  shall  also  use  the 
generating  function  methods  developed  in  Unit  7  and  applied  to  the  simple  birth- 
death  process  in  Section  3  of  this  unit,  to  find  the  distribution  of  X{t),  the 
population  size  at  time  t.  You  will  need  paper  to  write  on  as  you  work  through 
this  audio-tape  session. 

Now  switch  on  the  tape. 


The  immigration-death  model 

Immigration :  Individuals  join  the  population  according  to  a  Poisson 
process  at  rate  A . 


Death: 


The  lifetime  distribution  of  each  individual  is 
exponential,  M  (v). 


The  overall  birth  and  deaf h  rates  of  this  process  are 


h  = 


* 


Vv  = 


The  Kolmogorov  forward  equationsare 
^-px(0  =  ftx-\  Px-i(t)  +  vx+1px  +  1(t)  -  (ftx  +  vx)px(t),  x  =  0,1, 
so  the  differential-difference  equations  for  the  immigration-death  model  are 


>">  - 

■ 


The  equilibrium  distribution 

The  equilibrium  distribution,  if  if  exists,  is  foand  by  puffing 
p*(0  =  0  in  the  forward  equations,  and  solving  them. 


For  x  =  0, 


For  x  = 


For  x  =  Z, 


For  general  a, 


so  pt 


=  0,  sop2  = 

=  0,  so  p^  = 


,  x  =  1,2,..., 
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For  a  proper  probability  distribution,  £  px  =  l. 

x=0 

oo  oO  |  I  , _ , 


L  px  =  po  L 

x=0  x=0  - 

The  equilibrium  distribution  is 

P*  = 


,  50  po  = 


*  =  0,1,..., 


and  this  is  a 


distribution  with  parameter 


^4 J  The  partial  differential  equation  for  the  p.g.f.  TT (s,t) 

.  1  oo 

The  p.g.f.  of  X(t)  is  TTfs.t)  =  E  px(f)sx,  so 


Using  the  forward  equations  gives 

OO  00 

E  4rPx(t)sX  =  AE  Px-l(t)5X  +  v 
c  =  0  ^  x=1  - 

So  the  partial  differential  equation  for  TT(s,t)  is 
=  XsTT  +  v  -X  -v 


Solution  of  the  partial  differential  equation  to  find  7T  (s,  t) 

In  Lagrange  form  the  partial  differential  equation  is 

I  3TT  ,  [  I  9TT  I - 1  X 


Step  I 


so  the  auxiliary  equations  are 

ds  dt  dTT 


Step  2 


This  gives  a  trio  of  equations  : 


ds 

i 

i 

Ts 

T5 

11 

i 

dTT 

ds 

dTT 

J  -  - 

1  [ 
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3  Solve  the  auxi Wary  equations. 


»0-s) 


-  -1  dt 


-  vt  +  cons\ar\\ 


x  constant. 


The  first  solution  is 


+  constant 


or  TT  = 


x  constant. 


The  second  solution  is 


C)  — 


The  general  solution  is  obtained  from  c?  =  ^(ct);  that  is, 

TTe~Xs/l>  =  T//((l-s)e~vf), 
so  TT  (s,  t)  =  eAs/v  ilr  ((l-s)e_vf). 


-s  e 


5tep  4 


The  particular  solution  is  obtained  by  using  the  initial  condiYm. 

If  X(0)  =  0,  thenTT(s;0)  =  .  Putting  t  =  0  in  the  genera, 

solutiongives 


As/u 

=  e  y 


Step  5 


To  find  ^(x),  put  x  =  ;  then  s  =  .  This  gives 


fW  = 


TT(s, t)  =  e' 


=  exp 


This  is  the  p.g.f.  of  a 


distribution  with  parameter 


As  t  co 


TT(s,t) 


so  the  equilibrium  disfrib ution  is  a 


distribution  with  parameter 
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If  there  are  x0  individuals  at  the  start, then 
X(t)  =  X1(t)  +  X2(t), 

where 

(1)  XT (t)  is  the  number  alive  at  time  t  in  a  pure  death  process 
starting  with  x0  individuals; 

(2)  X2(t)  is  the  number  alive  at  timet  in  an  immigration-death 
process  starting  with  0  individuals. 

Then  Xt(t)  -  B(x0,  e~vf),  C^Un 


X2(t)  ~  Poisson  (l-e  vf)j  . 

As  t  ->  oo,  Xj(t)  — ►  0  and  X2(t)  ~  Poisson  ,  so  the 


Unit  7  Section  5 


equilibrium  distribution  is  Poisson  ^  . 

v  / 


iJA)  The  overall  birth  and  death  rates  are 


ySx  =  |_^J ,  Vx  =  ■ 

The  differential-difference  equafionsare 

=  PT-iO)  +  v(x  +  Opx+iO)  ~  (A  +  vx)px(t)  | ,  x  = 

PoCO  =  vpi(t)  -Apo(t)  . 


,2,..., 


so  p, 


For  x  =  0,  vpt  -  Xp0  =  0, 

For  x  =  I,  Ap0  +  2yp2-(A  +  »)pi  =  0, 

but  vp,  =  Ap0  so  Zvpz  -  Api  =  0, 
Forx=Z,  kp]  +  3vp}  -(A+2v)p2  =  0, 


but  2vpz  =  Ap,  so  3vp3  -  Ap2  =  0,  so  p3=  j,  (^)3  p, 


_  _A_  JL 

sop2-  Zv  Pi  -  Zvz  p0 


For  general  x, 


n  _  XI AY 

P*  x!  (  v  J  P«- 
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For  a  proper  probability  distribution,  £  px  =  1. 

\=o 

oo  oo  T  r^n 

F  p*  =  p0  F  ryFr)  =  p°  jTF  - S( 


_  -A/v 


The  equilibrium  distribution  is 


_  *-Vw  I _A_ 


Px  =  e 


Po  e _ ,  50  p0  =  e 


x  =  0,1,..., 


and  this  is  a  Poisson  distribution  with  parameter  — 


Thep.g.f.  of  X(t)  is  TT(s,t)  =  £  px(f)sx,  so 

x=  0 


M  _  y  ,.s  : 
9s  "  £0xP*(t,s 


M_  f  d  : 

9t  x=odtPx(t)s 


Using  the  forward  equations  gives 

"  ,  »  ps  ps - 

L  J-pxWs*-  A  I  px_,(f)sx  +  v  E(x+1)px+i(t)sx  -X  F  px(t) 


X 

S  -v 


uu 

LxdJ\) 


Lx  px< 

X  =  1 


00  I  00  00 

=  A  E  PxO)sx+  +  v  Z  xpx(\)sx~]  -  A  £  px(t)sx  -  v  £  xpx(f)sx. 

x=0  x  =  0  x  =  0  x  =  0 

So  the  partial  differential  equation  for  TT(s,f)is 


=  XsTf  +  v  ^  -k  IT  -vs 


=  -A(I-s)|tT  +  |v(1-s)|  ~  . 


ISA)  In  Lagrange  form  the  partial  differential  equation  is 

Wl-5)|f +  R|f.|x(1-s)Tr|f 


J/i  Os  l _ LI  at  “  LiTf 

so  the  auxiliary  equations  are 

ds  =  _dt_  =  dTT 

v(l-s)  -1  A(1-s)TT 
-  - 

.This  gives  a  trio  of  equations : 


— 

ds  __  dt 

dt  dTT 

ds 

dTT 

p(1-s)  -1 

-1  A(l-s)TT 

\v{X^gJ 

A£T^TT 
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5.2  Immigration,  birth  and  death 

We  have  now  considered  all  cases  where  only  one  or  two  of  immigration,  birth 
and  death  occur  to  change  the  size  of  a  population.  The  analysis  when  all  three 
occur  together  introduces  no  further  new  concepts  and  is  developed  here  as  a 
series  of  questions.  You  may  wish  to  study  this  section  later  and  use  it  as  revision. 

Firstly,  consider  in  general  terms  what  are  the  possible  forms  of  behaviour  of  the 
simple  immigration-birth-death  process.  You  can  deduce  much  of  this  from  the 
behaviour  of  the  simpler  models  you  have  already  studied. 
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Question  5.1  Discuss  the  types  of  behaviour  that  are  possible  for  the  birth,  death 
and  immigration  process.  In  particular,  consider  whether  extinction  is  possible  and 
what  happens  to  the  distribution  of  population  size  as  t  ->  co.  □ 


We  shall  now  investigate  the  mathematical  properties  of  the  model.  The  overall 
birth  rate,  fix,  contains  two  terms,  one  arising  from  the  possibility  of  any  of  the  x 
individuals  giving  birth  and  the  other  from  the  possibility  of  the  arrival  of  a  new 
individual.  And  the  death  rate,  vx,  derives  from  the  possibility  of  any  of  the  x 
individuals  dying.  So  we  have 

px  =  fix  +  2,  vx  =  vx. 

Using  the  Kolmogorov  forward  equations  (1.7),  we  can  write  down  the  forward 
equations  for  this  process.  For  x  =  0, 1,  ...,  we  obtain 

jtPx(t)  =  (P(x  -  1)  +  2)pJC_1(f)  +  v(x  +  \)px+1(t)  -  ((/?  +  v)x  +  X)px(t).  (5.1) 

As  usual,  px{t)  denotes  the  probability  that  the  population  contains  x  individuals  at 
time  t.  In  Subsection  1.3,  p-^t)  was  defined  to  be  zero,  so  when  x  =  0, 

Equation  (5.1)  takes  the  form: 

jtPo(t)  =  vpy(t)  -  XpQ(t). 

There  are  two  possible  ways  in  which  we  may  proceed.  We  can  determine  the 
equilibrium  distribution  directly  by  setting  ~  px(t)  equal  to  zero  in  the  forward 

Ui 

equations  and  solving  for  px  the  equations  thus  obtained.  Alternatively,  we  can 
find  the  p.g.f.  of  X(t),  the  population  size  at  time  t,  and  then  find  the  equilibrium 
distribution,  if  it  exists,  by  letting  t  ->  co  in  the  p.g.f.,  and  identifying  the  p.g.f. 
obtained  in  the  limit.  We  shall  pursue  the  latter  course.  At  some  stage  you  might 
like  to  find  the  equilibrium  distribution  directly  and  verify  that  the  results  obtained 
by  this  method  are  the  same. 

Question  5.2  Use  the  forward  equations  (5.1)  to  derive  the  partial  differential 
equation  for  n(s,  t),  the  p.g.f.  of  X{t),  and  write  this  equation  in  Lagrange 
form.  □ 


You  should  have  obtained  the  following  partial  differential  equation  for  the  p.g.f. 
of  the  immigration-birth-death  process: 

(1  -  s)(v  -  ps)^-~  =  X(1  -s)n.  (5.2) 

This  equation  includes  the  partial  differential  equations  for  all  processes  involving 
some  combination  of  birth,  death  and  immigration.  If  one  or  two  of  birth,  death 
and  immigration  are  not  included  in  a  model,  one  simply  writes  fi,  v  or  2  equal  to 
zero,  as  appropriate,  in  Equation  (5.2). 

We  shall  first  solve  Equation  (5.2)  for  the  case  fi  i-  v  and  discuss  this  model;  the 
case  fi  =  v  is  considered  later.  We  shall  follow  the  steps  of  the  procedure  for 
solving  Lagrange’s  equation.  The  equation  is  already  in  Lagrange  form,  so  the  next 
step  (Step  2)  is  to  write  down  the  auxiliary  equations: 

ds  _  dt  _  dYl 

(1  -s)(v-  fis)  2(1  -s)II'  (5,3) 

The  first  two  expressions  do  not  contain  2,  so  they  are  identical  to  those  in  the 
simple  birth-death  process.  So  of  course  the  solution  is  the  same!  This  was  derived 
in  the  solution  to  Question  3.3: 

1  1  -  s  e  ■ 

A  second  solution  of  the  auxiliary  equations  is  obtained  by  taking  the  third 
expression  with  one  of  the  first  two. 


Question  5.3  Obtain  a  second  solution  of  the  auxiliary  equations  (5.3).  □ 


33 


This  completes  Step  3  of  the  procedure.  Step  4  involves  writing  the  general 
solution  in  a  form  containing  an  arbitrary  function;  and  this  function  is  found  in 
the  final  step  by  using  the  initial  condition.  We  shall  do  this  first  when  the 
population  contains  no  individuals  at  time  t  =  0. 


Question  5.4 

*(i)  When  P  #  v,  obtain  the  general  solution  of  the  partial  differential  equation  (5.2) 
in  the  form  of  an  expression  for  II(s,  t ). 

(ii)  If  the  population  contains  no  individuals  at  time  t  =  0,  determine  the  form  of 
the  arbitrary  function  in  the  general  solution  found  in  part  (i),  and  hence 
obtain  an  expression  for  the  p.g.f.  of  X(t).  □ 


When  the  population  starts  with  zero  individuals  and  /?  #  v,  the  p.g.f.  takes  the 
rather  complicated-looking  form: 

A_(  (v  -  p)e(v~p)t  \,p 

\(v  -  psy'-n  -  p(i  -  s)J  '  (5-4) 

Notice  that  the  immigration  rate,  A,  occurs  only  as  the  index  to  the  expression  in 
brackets.  And  if  A  =  0,  then  n(s,  t)  =  1,  as  we  should  expect — if  no  immigration 
takes  place  then  the  population  size  must  remain  zero. 

By  writing  a  =  e(v~p)t  and  rearranging  the  expression  in  (5.4),  the  p.g.f.  of  X(t)  can 
be  rewritten  in  the  form 


n(s,  t)  = 


(v  -  P)a/(va  -  p) 

1  -  sp{ a  -  l)/(va  -  P) 


MP 


Now 


(v  —  P)cc  jg(a-l)_ 
va  —  p  vol  —  p  ’ 

and  so,  writing  r  =  X/p,  p  =  (v  -  /j)a/(va  -  P)  and  q  =  1  -  p,  we  have 

So  the  distribution  of  the  population  size  at  time  t.  has  a  negative  binomial  form, 
with  r  greater  than  zero  but  not  necessarily  an  integer.  Notice  that  if  v  =  0,  then 
P  =  e  i  the  p.g.f.  reduces  to  that  obtained  in  Unit  7  for  the  immigration- 
birth  process,  but  with  x0  =  0. 


Now  the  mean  and  variance  of  a  negative  binomial  distribution  are  given  by  the 
same  formulas  whether  or  not  r  is  an  integer:  p  =  rq/p,  a2  =  rq/p2.  So  the  mean 
and  variance  of  X(t)  can  be  written  down  directly  using  these  formulas,  giving 


ElX(t)l  = 


A(1 


v-p 


(5.5) 


KW0]=^L-g'(V'g)<)(v-^-(V-^) 

(v  -  P)2 

These  results  can  also  be  obtained  directly  from  the  p.g.f.  by  differentiating  U(s,  t) 
partially  with  respect  to  s  and  using  the  formulas  for  the  mean  and  variance  of  a 
random  variable  involving  the  derivatives  of  the  p.g.f.  at  s  =  1. 


We  now  consider  the  case  p  =  v;  that  is,  where  the  individual  birth  and  death  rates 
are  equal. 


Question  5.5  When  p  =  v,  the  partial  differential  equation  (5.2)  for  II(s,  t)  takes 
the  form 


an 

dt 


(i)  Solve  this  equation  when  X(0)  =  0. 

(ii)  Show  that  the  distribution  of  X(t)  is  negative  binomial,  and  obtain  its  mean 
and  variance.  □ 


We  now  have  all  the  results  we  need  to  compare  the  behaviour  of  the  process  for 
P>v,p  =  v  and  p  <  v.  From  Result  (5.5),  the  expected  population  size  grows 
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exponentially  for  large  t  when  p  >  v;  and  when  p  <  v  it  tends  to  a  limiting  value 
A/(v  —  p).  From  Solution  5.5,  the  expected  population  size  is  equal  to  X t  for  all  t 
when  /J  =  v.  Thus,  for  large  r,  we  have 

E[X(t)-]  ~  AeV-y»/{0  -  v),  when  p  >  v, 

£[Y(r)]  =  Xt,  when  p  =  v, 

E[X(t)-]  ~  A/(v  -  P),  when  p  <  v. 

This  indicates  that  the  population  size  is  expected  to  increase  without  bound  for 
P  >  v,  but  that  it  is  expected  to  remain  bounded  for  /j  <  v.  To  investigate  the 
behaviour  of  the  process  in  more  detail,  we  return  to  the  p.g.f. 

Question  5.6  For  each  of  the  cases  p  <  v,  p  =  v,  p  >  v,  find  the  limit  as  t  -+  oo,  if 
it  exists,  of  the  p.g.f.  of  the  immigration-birth-death  process.  Hence  describe  the 
limiting  behaviour  of  the  process  for  each  of  these  cases.  □ 

Finally,  we  shall  consider  the  situation  where  the  population  contains  x0 
individuals  at  time  t  =  0.  Although  this  appears  to  be  much  more  complicated 
than  when  there  are  no  individuals  initially,  the  general  solution  can  be  found 
easily  from  results  already  obtained. 

Question  5.7 

(i)  Express  X(t),  the  number  of  individuals  at  time  t  in  an  immigration-birth- 
death  process  containing  x0  individuals  at  time  t  =  0,  as  the  sum  of  two 
independent  random  variables. 

(ii)  State  the  distribution  of  each  of  these  random  variables,  and  hence  find  the 
mean  number  of  individuals  alive  in  the  immigration-birth-death  process  at 
time  t.  Distinguish  between  the  cases  P  =  v  and  P  #  v. 

(iii)  When  p  <  v,  find  the  limiting  distribution  of  the  immigration-birth-death 
process  starting  with  x0  individuals.  □ 

This  completes  our  study  of  Markov  population  processes  where  the  only  changes 
possible  are  due  to  birth,  death  and  immigration.  Tn  Unit  11  we  shall  return  to 
population  processes,  and  look  at  some  non-Markov  models  where  the  individual 
birth  rate  depends  on  the  age  of  the  individual  or  on  the  total  population  size. 
However,  in  the  next  unit  we  shall  look  at  queues,  which  may  be  thought  of  as 
Markov  population  processes  in  which  emigration  may  occur.  This  means,  in  the 
language  of  population  processes,  that  individuals  may  leave  the  population 
according  to  a  Poisson  process.  You  will  see  that  this  introduces  an  extra 
complication. 


Objectives 


After  studying  this  unit  you  should  be  able  to: 

explain  what  is  meant  by  a  Markov  process  and  a  birth  and  death  process; 

derive  the  Kolmogorov  forward  equations  for  a  birth  and  death  process,  and  use 
these  equations  to  obtain  the  differential-difference  equations  for  specific  processes; 

derive  simple  properties  of  a  two-state  Markov  process,  and  calculate  probabilities 
of  events  associated  with  this  process; 

calculate  the  p.g.f.’s  for  the  simple  birth-death  process,  the  immigration-death 
process  and  the  immigration-birth-death  process,  using  the  procedure  for  solving 
Lagrange’s  equation  which  was  described  in  Unit  7; 

use  these  p.g.f.’s  to  derive  properties  such  as  the  probability  of  extinction  and  the 
distribution  of  the  population  size  at  time  t; 

use  the  random  walk  and  the  Galton-Watson  branching  process  embedded  in  a 
simple  birth-death  process  to  obtain  properties  of  the  process; 

derive  the  equilibrium  distributions  of  the  immigration-death  process  and  the 
immigration-birth-death  process. 


Appendix:  Solutions  to  questions 


Section  1 

1.1  (i)  In  a  pure  death  process,  if  there  are  x  individuals  in 
the  population  at  time  t,  then  the  probability  of  a  death  in 
the  interval  [ t ,  t  +  <5t]  is  vx  St  +  o(St).  This  does  not  depend 
on  the  history  of  the  process  before  time  t;  the  most  recent 
death  might  have  occurred  at  any  time  before  t.  Hence  this  is 
a  Markov  process. 

(ii)  In  this  process  the  probability  that  an  event  occurs  in 
[r,  t  +  St ]  depends  on  t,  but  is  independent  of  the  number  of 
events  that  have  occurred  up  to  time  t  and  of  when  these 
events,  if  any,  occurred.  So  this  process  possesses  the  Markov 
property. 

(iii)  For  this  point  process  the  probability  that  an  event 
occurs  in  [t,t  +  St]  depends  on  how  long  before  t  the 
previous  event  occurred.  If  it  occurred  less  than  one  unit  of 
time  before  t  then  the  probability  is  zero.  On  the  other  hand, 
if  the  previous  event  occurred  at  time  t  -  2.9,  then  the  next 
event  is  certain  to  occur  in  the  next  0.1  unit;  so  the 
probability  that  it  will  occur  in  the  interval  [t,r  +  St ]  is  10  dr. 
Hence  the  probability  of  occurrence  depends  on  the  history 
of  the  process  before  t,  and  consequently  this  is  not  a 
Markov  process. 

1.2  In  the  pure  death  process  each  individual, 
independently  of  other  individuals,  has  probability 

vSt  +  o{St)  of  dying  in  the  interval  [r,r  +  dr],  and  there  are 
no  births.  Hence  ()x  =  0,  v*  =  vx. 

1*3(0  If  Px  =  A,  v*  =  0,  then  the  probability  that  A'(r) 
increases  by  one  in  [r,r  +  dr]  is  A  St  +  o(St),  and  it  is 
impossible  for  X(t)  to  decrease.  So  X(t)  is  the  number  of 
events  to  occur  by  lime  t  in  a  Poisson  process. 

(ii)  If  fix  =  A{t),  v*  =  0,  then  the  probability  of  a  birth  in 
[r,  t  +  dr]  is  A(t)  St  +  o(St),  a  function  of  r.  As  in  part  (i), 
deaths  are  impossible.  This  process  is  a  time-dependent 
Poisson  process. 

l*4(i)  For  the  Poisson  process,  fix  =  A,  vx  =  0,  so, 
remembering  p_l(t)  =  0,  Equations  (1.7)  become,  for 
x  =  0, 1, ..., 

jtPx(t)  =  *px-l(t)~  Apx(t). 

These  are  the  differential-difference  equations  for  the  Poisson 
process. 

(ii)  For  the  simple  birth  process,  Px  =  fix,  vx  =  0,  so  the 
Kolmogorov  forward  equations  are,  for  x  =  1,2,  ..., 

jtPx(t)  =  P(x  -  1  )px-1(t)  -  pxPx{t). 

These  are  Equations  (2.4)  of  Unit  7.  There  is  no  need  to  give 
the  equation  for  x  =  0,  as  the  simple  birth  process  must  start 
with  at  least  one  individual,  so  p0(t)  =  0. 


Section  2 


2.1(i)  The  time  7]  is  the  sum  of  two  independent  variates: 
the  time  the  machine  runs,  which  has  an  exponential 
distribution,  JVf(v),  and  the  repair  time,  which  has  an 
exponential  distribution,  M(A).  Hence 


£(T,)=i  +  i 

A  V 


vm  = 


a2 


(ii)  The  time  W„  can  be  written  as 
Wn  =  Tx+...  +  Tn, 

where  7], ...,  T„  are  n  independent  variates  each  with  the 
same  distribution,  and  hence  the  same  mean  and  variance,  as 
Tx.  So 

£W)  =  y  +  ”  V(Wn)  =  -^  +  ^. 

A  v  A2  v2 


2.2  If  the  process  is  initially  in  state  1,  then  plt0(0)  =  0  and 
Pi.i(0)  =  1.  Hence,  putting  t  =  0  in  Equation  (2.3)  leads  to 


so  that 

v 

c  —  —  - - , 

A  +  V 

and  hence 


Pi,o(0  = 
Pi.i(f)  = 


A  +  v 
A 

A  +  v 


(1  -  e~u+v)t), 

-I- _ v  c~g.  +  \)t 

A  +  v 


2.3  Using  Formula  (2.4a)  and  Solution  2.2,  we  obtain  the 
following  table  of  probabilities. 


t 

0.2 

0.5 

1 

2 

4 

Po.oW 

0.827 

0.648 

0.482 

0.367 

0.335 

Pl.o(0 

0.086 

0.176 

0.259 

0.317 

0.333 

2.4  From  Solution  2.2,  we  have 
v 


P  i.o(0  —  ~ 

Pi.iW  = 


A  +  v 

7. 


(1  -  e-<•^+v,,), 


+  - 


A  +  v  '  /  +  v 
As  t  -*  oo,  e~a+v)l  ->  0  and  hence 
v  A 


P  i.o(0 


P i.i(f) 


A  +  v’  +  v 

These  are  the  same  as  the  limiting  probabilities  when  the 
process  starts  in  state  0. 


2.5(i)  For  a  two-state  Markov  process  {2f(t);  t  >  0}, 
px(t)  =  P(X(t )  =  x) 

=  P(X(t)  =  x  |  A'(O)  =  0)  P{X(0)  =  0) 

+  P[X(t)  =  x  |  X(0)  =  1)  P(X(0)  =  1) 

=  a0p0,x(t)  +  alPlx(t). 

Now  as  t  — »  co,  p0Jt)  -*■  px  and  plx{t)  ->  px,  so 
Px(t)-*“oPx  +  Px  =  Px- 

Thus  the  limiting  probabilities  are  the  same  whatever  the 
distribution  of  the  initial  state  of  the  process;  and  so  the 
process  has  the  limiting  distribution  {px}  for  all  values  of  a0 
and  a, . 


(ii)  If  a0  =  v/(A  +  v),  and  hence  ay  =  A/{A  +  v),  then,  from 
part  (i),  p0(t),  the  probability  that  the  process  is  in  state  0  at 
time  t,  is  given  by 

Po{t)  =  +  JT^Puoit)- 

So,  using  Solution  2.2  and  Formula  (2.4a),  we  have 

Po(t)  =  —  1  ,2(v2  +  vAe~u+v)t  +  Av  -  2ve“(2+v)') 

(A  +  v)2  ’ 


A  +  v‘ 

And  hence 


Pi(0  =  1  -  P0(f)  = 


A  +  v‘ 
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2.6  For  each  cow  standing  at  time  t  =  0,  the  probability 
that  it  is  standing  at  time  t  is  p14(t),  so  the  expected  number 
of  these  cows  standing  at  time  r  is  apui(t).  Similarly,  of  those 
chewing  the  cud  at  time  t  —  0,  the  expected  number  standing 
at  time  t  is  (n  —  a)p01(t). 


Hence  the  expected  number  of  cows  standing  at  time  t  is 
given  by 


a( jj-  +  j^-e-^+A  +  ^(1  -  e~^') 

\A  +  v  2  +  v  J  X  +  v 


=  ae~{X+v) '  +  -^-(1  -  <TW+V)') 
X  +  v  ' 


nX  nX  —  a(X  +  v) 


X  +  v  X  +  v 

Notice  that  this  tends  to  nX/(X  +  v)  as  t  —>  co,  as  one  would 
expect. 


Section  3 


3.1  Multiplying  by  sx  and  summing  over  x,  we  obtain 


Z  -t:P*(0sx=  Z  P(x-  1  )px-1(t)s* 


+  Z  v(x  +  l)px+i(t)sx 

x  =  0 


-  Z  (P  +  v)xpx(t)sx. 


Rewriting  this  so  that  each  term  contains  px(t),  we  have 

00  //  «>  co 

Z  JtPxWsX  =  P  Z  xPx(t)sx+1  +  V  Z  xpjfis*-1 


-  (P  +  V)  Z  xpx(t)sx. 


Note  that  when  the  summand  contains  x,  it  is  irrelevant 
whether  the  lower  limit  is  0  or  1,  since  the  summand  is  zero 
when  x  =  0. 


nM  =  Z  Px(t)sx, 

x  =  0 


du  -  $  d  ,A  x  d n  »  ..  x_. 

dt  ^0dtPxi)s’  ds~^xPx^s  • 
So  the  equation  for  H(.s,  t)  is 


^  =  (fr2  +v-(/l  +  v)s)®. 


3.2(i)  Rewriting  Equation  (3.2)  in  Lagrange  form  we  have 


^_(/i52  +  v-(/?  +  v)s)f  =  0, 


so  /=  —  (ps2  +  v  -  (/i  +  v)s),  g  -  1  and  h  --  0. 
(ii)  The  auxiliary  equations  are 

_ ds _ _dt  _d  n 

-{/is2  +  v  -  (/?  +  v)s)  “  T  _  T' 


3.3(i)  Since  h  =  0,  we  can  write  down  one  solution 
immediately: 

cx  -  n. 

Taking  the  first  and  second  expressions  together  gives 
f  —ds  _  dt 

J  /is2  +  v  —  0 3  +  v)s  ,  1  ’ 


1  /  1 


—  V  \1  —  S  V  —  /is 


ds  =  1  dt. 


It  is  preferable,  at  this  stage,  to  write  1  -  s  rather  than  s  —  1, 
since  a  p.g.f.  is  defined  for  |s|  <  1,  and  in  this  range  1  -  s  is 
positive.  Now  doing  the  integration  we  have 

~p~_  v(~  log (1  —  s)  +  log(v  —  /is))  =  t  +  constant, 


(  v  —  Rs 

log  I  j  _  s  J  =  {P  —  V)t  +  constant. 


giving 

v  —  Ps  in- 

- - =  ew  v>i  x  constant 

1  -  s 

Hence  a  second  solution  is 
1  —  s 


(ii)  The  general  solution  is  given  by  cx  =  i//(c2);  that  is, 


\\{s,t)  =  MV-—^-e-^-v)t 

1  -  5 


3.4(i)  If  there  is  one  individual  at  time  t  =  0,  then 
n(s,0)  =  s.  So,  putting  t  =  0  in  the  general  solution  found  in 
Solution  3.3,  we  have 


v  -  [is 


Writing  x  =  (v  -  /is)/(l  -  s),  so  that  s  =  (v  -  x)/(/i  -  x),  we 
find 


•AM  = 


P-x 

Hence,  from  Solution  3.3(ii),  we  have 


=  v(l  -s)-(v-/ls)g<v-^ 

’  P(l  -  s)  -  (v  -  Ps)elv~p)t' 


(ii)  If  there  are  x0  individuals  at  time  t  =  0,  then 
n(s,0)  =  sx°.  Putting  t  =  0  in  the  general  solution  gives 

_  J v  -  Ps\ 


sx°  =  i p 


Continuing  as  in  part  (i)  by  writing  x  =  (v  —  /is)/(l  -  s)  leads 
to 


<A(x)  = 


and  hence 


H  (s,t)  = 


v(l  —  s)  —  (v  —  Ps)e(v 


-  s )  -  (V  -  Ps)e(v~B),J  ‘ 


3.5(i)  Written  in  Lagrange  form,  Equation  (3.4)  is 


so  /=  ~P(  1  —  s)2,  g  =  1,  h  =  0.  The  auxiliary  equations  are 
ds  _  dt  _  d  n 
-P(l  -s)2_T“^0"' 

Since  h  =  0,  one  solution  is  c,  =  n.  A  second  solution  is 
obtained  from 

ds  dt 
-/J(1-s)2_7; 
that  is,  from 

Im. 


d  -  s)2 


—  Pt  +  constant 


c2  =  Pt  + 


The  general  solution  is  given  by  c,  —  i//(c2);  that  is. 


n(s,  t)  =  1 H  pt  + 


1  —s' 
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(ii)  If  there  are  x0  individuals  at  time  t  =  0,  then 
II(s,0)  =  s*0,  and  so 


Writing  x  =  1/(1  —  s),  so  that  s  —  (x  —  l)/x,  leads  to 


so 


n(s,t)  = 


pt  + 


1  —  s 


- 1 


Pt+T h 

jit  —  sfh  +  s\° 
jit  —  sjit  +  1 


.  0,..  D  .  a  —  bs  it  -  s(  k  —  1) 

3.9(i)  By  comparing - r  and  - - we  find 

c  —  as  1  +  jit  —  sjit 

a  =  fit,  b  —  jit  —  1,  c  =  1  +  jit,  d  —  jit.  So  ad  —  be  =  1  and 
c  —  d  =  1,  and  hence  from  Solution  3.7(i),  the  mean  is  given 
by 


and  the  variance  is 


(c  -  dy 
=  2d  =  2 jit. 


(ii)  If  the  process  starts  with  x0  individuals,  then  the  mean 
and  variance  are  given  by 

W  =  x  o, 

VlX(t)]  =  2x0fit. 


Setting  s  =  0  in  Formula  (3.5)  gives 
fit 


3.6(i) 

this  is  the  probability  that  extinction  has  occurred  by  time  t. 

(ii)  Letting  t  ->  co  in  Formula  (3.5)  for  the  p.g.f.  of  a  simple 
birth-death  process  with  /l  =  v,  we  obtain 

lim  n(s,«)  =  1. 

t~*  CO 

So  extinction  is  certain  to  occur  eventually. 

3.7(i) 

U'(s)  = 


n'(s)  =  (a  —  bs)/(c  —  ds),  so 
— (c  —  ds)b  +  (a  -  bs)d 


H "(s)  = 


(c  -  ds)2 
ad  —  be 
(c  -  ds)2' 

2d  (ad  —  be) 


(ad  —  be)2 


(c  -  ds)3 

Putting  s  =  1,  we  have 
ad -be 

a2  =  n"(l)  +  p  -  p2 

_  2d  (ad  —  be)  ad  —  be 

(c  -  d )3  +  (c  -  d)2  (c  —  d )4 

(ii)  Taking  a  =  v(l  -  p),  b  =  v  -  (ip,  c  =  ji  -  vp, 
d  =  [i(  1  —  p),  where  p  =  e(v~P)t,  we  have 
ad  -  be  =  p(v  -  ji)2, 
as  we  have  already  calculated,  and 
c-d  =  p(P  -  v). 

So  when  starting  with  a  single  individual,  the  mean  size  of 
the  population  at  time  t  is 


EtX(ty  = 


P(v  ~  Pf 

p\ji  -  V)2  p 


=  -  —  efp~v)t. 


The  variance  is  given  by 

P  (P  ~  v)3  p  p2 
2P(l-p)  +  p(P-v)-(P-v) 
p2((i  -  v) 
ji  +  V  —  pp  —  vp 


p2(P  -  V) 


p 


V  1/1  t 

p~vx  PL 


P  +  V 
P-V 


eu>  -  v)‘(eU>  -  v)I  _  J) 


3.10(i)  When  increases  and  decreases  in  population  size  are 
both  possible,  a  stochastic  model  is  based  on  a  pair  of 
equations, 

Px,x+i(t,t  +  St)  =  hi(x,t)dt  +  o(St), 
px,x-i(t,t  +  St)  =  h^Px,t)St  +  o(<5t), 
and  the  corresponding  deterministic  model  is  the  solution  of 
the  differential  equation 

^  =  Mz,t)  -  h-i(z,t). 

(See  Unit  7,  Equations  (4.4)  and  (4.5).) 

For  the  simple  birth-death  process,  h{(x,t)  =  Px  and 
b~ i (x,  t)  =  vx,  so  the  deterministic  model  is  the  solution  of 
the  differential  equation 
dz 

(ii)  Using  separation  of  variables  to  solve  this  equation,  we 
have 

Jr-K’)* 

so 

logz  =  (P  —  v)t  +  constant. 

But  z  =  z0  when  t  =  0,  so 
logz0  =  constant, 
and  hence 

log~  =  (P-  v)t, 

^0 

so 

z  =  z0  e<*~yU. 

(iii)  When  ji  <  v,  elp  ~ v)'  ->  0  as  t  -»  co ;  so  the  population 
decreases  in  size  but  never  dies  out  completely.  In  the 
stochastic  model  the  population  fluctuates  and  eventually 
becomes  extinct  with  probability  one. 

When  (i  >  v,  eUi~'n  ->  oo  as  t  ->  co,  so  in  the  deterministic 
model  the  population  grows  without  bound.  However,  in  the 
corresponding  stochastic  model  the  population  may  either 
grow  without  bound  with  probability  1  —  (v/P)z°,  or 
eventually  die  out  with  probability  (v/fi)z°. 

When  P  =  v,  e(P~v)l  —  1,  so  in  the  deterministic  model  the 
size  of  the  population  remains  at  z0.  But  in  the  stochastic 
model  extinction  is  certain  to  occur  eventually. 

Notice  that  for  all  values  of  P  and  v,  the  mean  of  the 
stochastic  model  is  equal  to  the  size  of  the  population  at 
time  t  in  the  analogous  deterministic  model. 


3.8  Since  X(t)  can  be  thought  of  as  the  sum  of  x0 
independent  variates,  each  with  mean  and  variance  as  found 
in  Solution  3.7(ii),  we  have 
ElX(t)-]  =  xoyp-^, 

vixw  =  x0  -  1). 


Section  4 

4.1(i)  The  probability  that  the  population  size  will  ever 
increase  to  k  if  there  are  initially  x0  individuals  is  equal  to 
the  probability  that,  in  the  gambler’s  ruin  problem,  Gary  will 
eventually  win  if  he  begins  with  £x0  and  he  and  his  opponent 
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have  £k  between  them.  So,  setting  p  =  /}/(/?  +  v), 
q  —  v/(P  4-  v),  j  =  x0  and  a  —  k,  the  required  probability  is 
given  by 


1 


-  = 


1  ~  (v/l)*° 

)  1  -  (v/Pf 

Xq 

k 


when  i=-  v 
when  P  =  v. 


(ii)  (a)  When  k  =  10,  x0 
probability  is  equal  to 


i-(i)5 

i-(f)10 


0.1164. 


5,  P  =  4  and  v  =  6,  this 


(b)  When  k  =  10,  x0  =  5,  P  =  6  and  v  =  4,  this  probability 
is 


i-(!)5 
i  -  (t)10 


=  0.8836. 


Section  5 

5.1  Since  there  is  immigration,  permanent  extinction 
cannot  take  place.  The  population  size  may  become  zero,  but 
eventually  a  new  individual  will  arrive  and  the  process  will 
continue.  For  a  simple  birth-death  process  with  no 
immigration,  extinction  is  certain  when  p  <  v,  but  if  p  >  v 
the  population  may  either  become  extinct  or  increase  in  size 
without  bound.  So  if  immigration  is  also  present  and  P  >  v, 
we  should  expect  that  eventually  the  population  size  will 
increase  indefinitely;  and  if  the  population  ever  dies  out,  then 
sooner  or  later  the  process  will  start  up  again  due  to 
immigration.  And  if  p  <  v,  we  should  expect  that  the 
distribution  of  the  population  size  will  approach  an 
equilibrium  distribution.  When  p  =  v,  it  is  not  obvious  what 
the  behaviour  will  be;  the  addition  of  immigration  prevents 
permanent  extinction  but  it  is  not  clear  whether  there  is  an 
equilibrium  distribution  for  the  population  size  or  whether 
the  population  will  eventually  increase  without  bound. 


4.2  The  diagrams  below  show  the  two  possible  ways  in 
which  exactly  three  individuals  exist. 


P( 0)  p(0) 


M0) 

The  probabilities  of  these  two  trees  are  p2(l)p(0)  and 
p{2)p2(0)  respectively.  For  the  branching  process  embedded 
in  a  birth-death  process, 

so  the  probability  that  a  total  of  three  individuals  exist  in  the 
simple  birth-death  process  is 

?ft2V3 

p2(l)p(0)  +  p(2)p2(0)  =  -/— - 

(P  +  v)5 


4.3(i)  When  P  =  v, 

F(t)  =  P(T  <  t) 

=  F(population  is  extinct  by  time  t ) 


from  Solution  (3.6). 


(ii)  As  t  -*  oo,  F(t )  ->  1,  so  F(t )  is  a  c.d.f.  and  hence  T  has  a 
proper  distribution.  The  p.d.f.  is  given  by 
x0  px°  tx°  ~ 1 


At)  =  F'(t)  = 


(1  +  Pt)x°' 


4.4(i)  The  expected  time  to  extinction  when  x0  =  1  is  given 
by 


F=\  (1  ~F(t))dt 

Pt 

1  +Pt 


J  o 


Jo 


dt 


1 


1  +  pt 


dt 


l 


-log(l  +  pt) 

Since  log(l  +  pt)  ->  oo  as  t  - 
time  to  extinction  is  infinite. 


co,  this  is  infinite,  so  the  mean 


(ii)  Extinction  is  certain  to  occur  eventually,  and  in  order 
for  the  population  to  become  extinct,  its  size  must  at  some 
stage  be  equal  to  one.  After  that  the  mean  time  to  extinction 
is  infinite.  So,  whatever  the  initial  size,  the  mean  time  to 
extinction  is  infinite. 


5.2  The  p.g.f.  of  X(t),  the  population  size  at  time  f,  is 
n  (s,t)  =  £  px(t)  sx, 
so 


x  =  0 


an  ^  d  x  eu  « 

Zj'PMf,  ~e 7  =  ^xp^t)s’ 

Multiplying  Equations  (5.1)  by  sx,  and  summing  over  x,  gives 

f  jtPx(t)sx  =  P  £  (x  -  l)px_,(r)sx  +  2  £  px-l(t)sx 


+  v  £  (x  +  1  )p  X  +  1  ( t)s* 


~  (P  +  v)  Z  xPx(t)sx  -  2  Z  Px(t)sx 

x=0  x=Q 

or,  changing  the  limits  so  that  px{t)  appears  in  the  summands 
instead  of  px-i(t)  and  p*+1(r), 

rT\  1X1  00 

~fa=P  Z  XPx(t)sX+1  +a£  px{t)sx+i 


»  ari 

+  vE  xpMs’-'-ifi  +  y^-XTL 


So 


„ ,an  .  „  an  ,  an 
_r  =  ^_  +  ASn  +  v__(^  +  v)s__An 

=  (!-»)(*-  -  -1(1  -  s)n. 

Note  that,  since  s  <  1,  it  is  always  advisable  to  write  1  —  s 
rather  than  s  —  1.  Written  in  Lagrange  form,  the  equation  is 

(i-snv-ps)f-f  =  x(i-s)n. 


5.3  If  we  take  the  second  and  third  expressions  together, 
the  equation  obtained  contains  s,  t  and  fl.  Since  all  three 
variables  are  involved,  a  solution  is  not  possible  by 
separation  of  variables,  so  we  take  the  first  and  third 
expressions  together.  Cancelling  the  term  1  —  s,  and  inserting 
minus  signs  on  both  sides  we  have 
-ds  -dU 

v  —  ps  m 

and  integrating  this  leads  to 

^log(v  —  ps)=  -ilogfl  +  constant, 

P  4 

or  * 


log  FI  +  ^  log  (v  —  Ps)  =  constant, 
which  may  be  rewritten  as 


c2  =  TT(v  -  ps)xlp. 
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5.4(i)  The  general  solution  is  c2  =  i[/(c x)  or 

n(v  -  pS)x/p  =  J'Lzlijy-to 
V  1  -  s 


n (s,t)  =  (v  -  ps)~x,piJ/(- — 

V  1  ~s 

(ii)  If  A'(O)  =  0,  then  po(0)  =  1,  p*(0)  =  0  for  x  #  0,  so 
Tl(s,0)  =  Y.PMsx=L 
Putting  t  =  0  in  the  general  solution  gives 


1  =  (v  - 


SO 


*\jz 7)  =  (v-^)w- 

If  x  =  (v  —  ps)/(l  —  s ),  then  s  =  (x  —  v)/(x  —  /?),  and  hence 


<AW  =  (  v  - 

Therefore 


P(x  ~  v) 

x  —  fi 


w 


X(V  -  P) 


X  — 


U(s,t)  =  (v  -  ps)  x,p 


XP 


\  l~s 
(v  -  P)elv~p>‘ 


(v  -  ps)e(v~p)t  -/i(l  -  S) 


5.5(i)  The  auxiliary  equations  are 
ds  _dt  _  dll 
-0(1  —  s)2  ~  1  “  -A(l  -  s)H ' 

Taking  the  first  and  second  expressions  together,  and 
integrating,  gives 

-1 

77: - r  =  t  +  constant, 

P(1  -s) 

so 


Cl  —  (It  +  - - . 

1  -  s 

Taking  the  first  and  third  expressions  together,  and 
cancelling  (1  -  s),  leads  to 
At/s  _  dU 

fll  -  s)  ~  If’ 

and  integrating  this  gives 

—  p  l°g  ( 1  —  s)  =  log  n  +  constant, 
or 


logn  +-log(l  -s) 


=  constant. 


so 


c2  =  n(l  -  s)x,p. 

Hence  the  general  solution  is  c2  =  iA(cj),  or 

n  =  n(s,  t)  =  a  -  S)-X'>^t  +  ji-Y 

Since  FI(s,0)  =  1,  we  have 

l  =  (l-s)-w^jd_| 

Setting  x  =  1/(1  —  s)  gives  t/t(x)  =  x~xip,  so 

1 

~  (1  +  0t  -  spt)x,p' 


(ii)  This  p.g.f.  can  be  written  as 


n  (s,o  = 


(  i/d  +  Pt)  T 

\i-m/(i  +  pt)j 


and  this  is  the  p.g.f.  of  a  negative  binomial  distribution  with 
r  —  HP  and  P  =  1/(1  +  Pt).  The  mean  and  variance  are  given 
by 


Emt)-]^  =  m  =  h, 
p  p 


vimi=r4=Mu-pt). 


5.6  When  P  ^  v. 


n(s,r)  = 


(v  -  p)e(v~P)t 


k(v  -  Ps)e^~P)t  -P(l-s) 
When  0  <  v,  e(v~p)l  -»  oo  as  t  ->  oo,  so 


n(s,  oo)  = 


v~  P 

v  —  ps 


=  (i-P/vY 

V  -  ps/v)  • 

In  this  case  an  equilibrium  distribution  exists  and  has  the 
negative  binomial  form. 


When  p  >  v,  e(v  ^  0  as  t->  oo,  so  n(s,  oo)  =  0.  No 

equilibrium  distribution  exists,  and  the  size  of  the  population 
increases  without  limit. 


When  P  =  v, 

->  0  as  t  -*  oo. 

Hence,  as  for  p  >  v,  there  is  no  equilibrium  distribution. 


5.7(i)  Since  all  individuals  behave  independently  of  each 
other,  immigration  will  have  no  effect  on  the  x0  individuals 
present  at  time  t  =  0.  Their  behaviour  will  follow  a  simple 
birth-death  process  which  contains  Xx(t)  individuals  at  time 
t.  Independent  of  this  process  is  an  immigration-birth-death 
process  beginning  with  no  individuals  at  time  t  =  0  and 
containing  X2{t)  individuals  at  time  t.  Hence  we  have 
X(t)  =  Xx  (t)  +  X2(t). 

(ii)  Xx(t)  is  the  sum  of  x0  variates  each  having  a  modified 
geometric  distribution  (see  Section  3).  When  p  #  v  the  mean 
of  X,(t)  is  x0e(p  v)‘,  and  when  p  =  v  the  mean  is  x0.  The 
variate  X2(t)  has  a  negative  binomial  distribution  with  mean 
given  by  Formula  (5.5)  when  p  #  v,  and  with  mean  It  when 
P  =  v,  from  Solution  5.5.  So  E[X(t)l  the  mean  number  of 
individuals  alive  at  time  t,  is  given  by 

£[X(f)]  =  fr'-"  +  ~  v  1  p - -  when/?#v 

[x0  +  At  when  p  =  v. 

(iii)  From  part  (i),  X{t)  =  X^t)  +  X2(t).  Since  p  <  v,  the 
population  of  the  simple  birth-death  process  starting  with  x0 
individuals  will  become  extinct  with  probability  1  as  t  ->  oo. 
So  the  limiting  distribution  of  X(t)  is  the  same  as  that  of 
AY2(t) — the  negative  binomial  distribution  given  in 
Solution  5.6. 
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